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1. EXECUTIVE SUMMARY 


1.1 Objective and scope 


The “Study on the quality of electricity market data of transmission system operators, 
electricity supply disruptions, and their impact on the European electricity markets” was 
carried out by VVA (lead), Copenhagen Economics, Neon and Deloitte for the European 
Commission (DG ENER) under the framework contract ENER/A4/516/2014. The study 
aimed to assess the completeness and quality of electricity market data reported by 
transmission system operators, identify and fill the data gaps related to electricity 
generation outages and supply disruptions as well as assess their impacts on consumers 
and on the market. The study covered the 28 Member States and the period 2010-2016. 


1.2 Methodology 


The study consisted of two work packages, both covering the 28 EU member states and 
involving data-intensive analyses, but independent from each other otherwise. 


Work package 1 (Outputs 1 and 2) aimed to evaluate the quality of the data provided 
by European TSOs through the ENTSO-E Transparency Platform? (TP) as well as the legal 
conditions upon which it can be used. Understanding to which degree market participants 
have access to this data is vital for facilitating efficient production, consumption and 
trading decisions as well as deeper market integration and the integration of variable 
renewable energy sources. 


For the technical evaluation (Output 1), an extensive statistical assessment was 
conducted, an online user survey was implemented, and experts were interviewed. The 
work package covers the period 2015-2016. For the legal evaluation (Output 2), a legal 
opinion was commissioned from a specialized lawyer. 


Based on these steps, we provided a detailed analysis of the completeness, accuracy, 
timeliness and technical accessibility (user friendliness) as well as legal accessibility of the 
data published in the ENTSO-E Transparency Platform, thus assessing the extent to which 
TSOs are fulfilling Commission Regulation (EU) 543/2013. 


Work Package 2 (Outputs 3 to 6) aimed to provide information on the electricity 
outages and significant electricity supply disruption events in the 2010-2016 period, with 
an analysis of their causes and their impact on the electricity market. We also assessed to 
what extent voluntary demand curtailments are used to face supply disruption events, 
and which supply margins they offer. Finally, we estimated the value of lost load resulting 
from the significant supply disruption events, representing the cost for society of these 
disruptions. 


This work package was articulated between desk-based research on the ENTSO-E 
platform and publications from national regulators, electricity providers, TSOs and DSOs 
on the one hand, and stakeholder consultation via web-based surveys or in-depth 
interviews to complete the information. 


1 https://transparency.entsoe.eu/ 


The identification of the main features and causes of electricity outages and supply 
disruption events as well as the use of voluntary demand curtailments and the value of 
lost load are essential to reduce the impact of such events and strengthen security of 


supply. 
1.3 Findings of the study and conclusions 


1.3.1 Work Package 1 
1.3.1.1 Analysis of ENTSO-E datasets (Output 1) 


Output 1 aimed to evaluate the quality of data published through the ENTSO-E 
Transparency Platform (TP) as well as the user friendliness of the platform itself. In the 
following, the core findings and conclusions are presented. 


Completeness 


Assessing completeness means verifying whether all data items specified in Regulation 
543/2013 are available on the TP for all geographic entities that apply and for all time 
steps since January 5, 2015, when the Regulation came into force. We identified two 
types of issues regarding completeness: 


e Missing data: data that should be published are not (data gaps) 
e Information about missing data: users are not informed about data gaps 


Additionally, some users requested broader coverage and identified data they would be 
interested in that go beyond what is prescribed in Regulation 543/2013. 


Our assessment confirmed that many of the data items are incomplete. For example, 
only one third of all countries have reported a complete time series of "Actual Total Load" 
and there is only a single case of a complete time series for "Day-ahead Prices". This is 
problematic as even just 1% of missing data can render the entire time series useless for 
many analyses. While most gaps in the data concern a few hours or days, some are 
stretching for an entire year. 


Moreover, the information of users about missing data is very limited. There is no 
overview of data completeness, a fact that complicates the use of TP data. 


Accuracy 


The accuracy analysis identifies whether data are "correct"; as a metric, we compared 
values on the TP to values reported elsewhere. We found four major issues related to 
accuracy of the TP: 


e Inconsistencies with other ENTSO-E data, 

e Inconsistencies with other data sources, 

e Information about inaccuracy: users are not informed about incorrect data and 
e Inaccurate data definitions. 


While inconsistencies between TP data and another source does by itself not imply that 
the TP data is incorrect (the inconsistency might stem from deviating data definitions, or 
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the other source might be wrong), we find the level of inconsistency considerable. 
Comparing “Actual total load” and “Aggregated wind generation per type” with data from 
Eurostat and ENTSO-E’s own Power Statistics, we find deviations for both data items for 
all countries, with differences ranging from minus 25% to plus 95%. The extent and the 
pattern of the deviations vary widely between countries and over time. 


As in the case of completeness, users are not informed about inaccuracies, nor can 
they inform other users if they identify problematic data. Finally, unclear data definitions 
lead to inaccurate data due to inconsistent interpretation by Data Providers (e.g., is a 100 
MW reporting threshold applied to individual machines such as a wind turbine or entire 
power plants such as a wind farm). 


Timeliness 


Assessing timeliness means confirming that data are published on the TP within 
reasonable timeframes, ideally those specified in Regulation 543/2013. We relied on user 
input for the timeliness assessment, in particular from market participants. We 
summarize the main issues with timeliness as follows: 


e Outage data and UMMs 
e Overwriting forecasts 
e Delays in data availability 


Timeliness is of particular importance for data used by market participants to inform their 
trading decisions such as outages of generation and consumption units as well as 
transmission grid elements, altogether referred to as "Urgent Market Messages" (UMMs). 
Outage data is one of the areas of particular concern: Users report that the UMM data 
published on the TP is burdensome to work with, as there are often duplicates 
or inconsistencies with other sources that also publish UMMs. Furthermore, 
users were missing an overview of all outages for a single asset. 


Another issue is that data are overwritten by updates even though the historical values 
may also be relevant for analysis. Users complained of useful historical data being 
overwritten by updates without any indication of whether there was an update, when it 
was made and where historical data would be available. Finally, users reported 
numerous examples of deadlines that were not adhered to. 


User friendliness 


We assessed user friendliness by means of an online survey and expert interviews as well 
as our own experience with the TP. Issues regarding user friendliness can be categorized 
into six broad categories: 


e Website and GUI, including ease of finding data, data presentation and ease of 
accessing data for downloads; 

e Automatic downloads, including useful documentation and reliable access; 

e Data files; 

e Displaying data availability, including why data is unavailable; 

e Data documentation and 


e User support. 


Regarding the website, the main issues are slow response times and impractical 
navigation that do not allow users to find relevant data and restrict download to one 
country at a time. A quickly accessible overview on data availability, showing which data 
items are expected for which countries would be valuable for users. Users are overall 
satisfied with the more advanced download options FTP and Restful API, but 
many are not aware these options exist or how to use them because of lacking 
information on the website. Availability and quality of data definitions is another issue. 


The TP does not publish contact details or even the identity of Data Providers or Primary 
Data Owners. All requests for user support are channelled through the TP service desk, 
which usually does not publish its replies, so future users are not warned about quality 
issues. Some users report waiting sometimes very long to get an answer to their 
requests. 


Suggestions for improvements 


The individual issues outlined in the sections on completeness, accuracy and timeliness 
should be improved upon. The objective should be to increase completeness of time 
series data such as load, prices and generation to 100%. Inconsistencies with other data 
sources should be resolved or explained. The provision of event-driven data, i.e. outage 
data (7.1.A&B, 10.1A&B, 15.1.A-D) should be improved due to its importance for market 
actors: the messages should be displayed in a sensible order, duplicates are to be 
avoided and older versions of UMMs should be retained and not overwritten. 


Furthermore, we developed the following suggestions for improvements that do not 
pertain to individual data items, but rather to cross-cutting issues related to usability, 
incentives and governance: 


e Improve information and navigation 

e Introduce a crowd-sourced public data quality log 
e Introduce automatic quality reporting 

e Publish machine-readable metadata 

e Adjust governance, ownership and incentives. 


1.3.1.2 |Terms on data management and access (Output 2) 


Output 2 concerned legal aspects of using the Transparency Platform (TP), in particular 
the applicability of intellectual property rights and implications of the terms of use of the 
TP. Specifically, it addressed the four questions below. In the following, the core findings 
and conclusions are presented. 


Is the TP protected under copyright or the sui generis database right? The TP is 
likely to be not protected by copyright, but under the sui generis database right 
established by Art. 7 of the Database Directive 96/9/EC. Therefore, it is protected as 
intellectual property. 
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What kind of use is currently legally permitted? To answer this question, for each 
type of use one needs to address three sub-questions: 


e Is the respective act of reusing the data covered by the exclusive rights of the 
right-holder (or is the use outside the scope of the protection)? When downloading 
data from the website of a Data Provider the person who downloads is creating a 
new copy of the data. This reproduction affects the rights of the owner of the sui 
generis right if the database is copied as a whole or a qualitatively or 
quantitatively substantial part of the database is copied. 

e Are any statutory exceptions to the sui generis right applicable (e.g. some kinds of 
private use, scientific research)? Some Member States permit by means of law 
downloading substantial parts for private or scientific use for non-commercial 
purposes. However, most use of TP is likely to be for commercial purposes; for 
these types of use, this exception does not apply. 

e Is the respective act of reusing the data allowed under the licence provided by the 
maker of the database? The “General Terms and Conditions for the Use of the 
ENTSO-E Transparency Platform” is governing the use of data published on 
ENTSO-E’s website. This is not a classical licence agreement but it stipulates which 
uses Shall be allowed. It does not include any explicit statement to grant the right 
to use the database. 


As a consequence, most use of the TP going beyond the mere download can be seen as 
constituting a copyright infringement. In other words, most users cannot use TP data ina 
legally safe manner. 


Who is the rightholder? It seems likely that both ENTSO-E and Primary Data Owners 
and possibly intermediaries that have transferred the data hold sui generis rights on the 
TP database. This is because it seems to be the case that they have made a qualitatively 
and/or quantitatively substantial investment in either the obtaining, verification or 
presentation of the contents. 


How can legally reliable reuse of TP data be granted in the future? The best way 
forward seems to be to grant an open licence of the TP database. This requires, however, 
consent of all rightholders. 


1.3.2 Work Package 2 


Security of electricity supply is a key concern for European Member States. A reliable 
power supply has indeed become one of the main foundations of modern economies. 
While overall, security of supply is thought to be high in the EU, concerns have been 
raised that it is now becoming challenged. This is due to a combination of the substantial 
- and increasing - deployment of intermittent renewable energy sources, and consequent 
closure of controllable capacity (also driven by an insufficient pricing of flexibility). These 
concerns have spurred a number of countries to improve the resiliency of the electricity 
system by e.g. improving the market design, pursuing subsidies for having adequate 
capacity available such as capacity payments and strategic reserves, and strengthening 
reserves for operational balancing (e.g. manual reserves). 


The reason for ensuring a well-functioning power supply is that power is a valuable 
resource to society. Every time a MWh is not supplied to a consumer that is willing to pay 
the price, this constitutes a loss to society. Such losses are typically measured by the so 
called ‘value of lost load’ (VoLL), which measures the perceived value to consumers of 
preventing an electricity disruption. 


In this study, we distinguish consistently between outages (focus in output 3) and 
disruptions (focus in output 4). We define an outage as an event where a power 
generation unit is out of service; this can be planned, for example for the maintenance of 
units, or forced for example as a consequence of technical malfunctions in the unit. An 
outage can lead to a loss of power for consumers but this will typically not be the case. 
We define a disruption as a situation where consumers cannot consume the amount of 
electricity they would otherwise have liked. A disruption is therefore associated with a 
loss of power to one or more consumers, e.g. through a brownout or local grid 
malfunctions. 


While the concern for security of supply is quite widespread across Member States, there 
is currently a lack of knowledge about the extent of disruptions across the EU and the 
overall loss to society from such disruptions. In addition, there seems to be no clear 
understanding of the causes of the disruptions, and in particular to what extent they were 
caused by not structurally having enough generation capacity available (called adequacy 
in the literature), or by problems related to grid infrastructure such as congestion and 
physical failure through e.g. damaged cables (called reliability in the literature).? 


In this study we are providing a more comprehensive picture of these issues. In particular 
we have been looking into the following questions: 


1. To what extent is European power generation capacity unavailable to the market, 
and is it due to planned maintenance or unplanned failures? What is the overall 
amount of 'non-generated energy' in the EU? 


2. What is the extent and causes of power disruptions? in the EU? 


3. To what extent has the demand side played a role in mitigating stressful situations 
in the system? 


4. What is the value lost to society from power disruptions? 


? The concept of reliability typically also includes the short term operational response by TSOs such as the use 


of Frequency Containment Reserves etc. 


? Defined as power not reaching consumers, which they would otherwise have paid to consume. 
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Our main take-away from the analysis is the following: 
We estimate that an average consumer in the EU faces one to two disruptions per 
year, each taking on average roughly 1-2 hours. In total, we come to the estimate 
that each year, approximately 600 to 850 GWh of electricity are not supplied to 
consumers that would otherwise have paid to consume it.* This amount of 
electricity corresponds roughly to the electricity consumption of Spain or Italy in one day. 
The main reasons for this are unplanned malfunctions in the grid - with the vast majority 
of malfunctions happening in the distribution grid. It is worth mentioning that there has to 
our knowledge been no major disruption in the transmission system in the period 2010- 
2016 leading to brownouts or blackouts. Transmission system disruptions can potentially 
have massive scale, and even a few of these evens would increase non-supplied 
electricity significantly. This can be exemplified by an event in Italy in 2003 where one 
massive event led to 56 million people being out of power for between 3-12 hours. This 
event alone led to about 180 GWh of power not being supplied to consumers.? 


The lost value to consumers from this non-supplied electricity is roughly 
estimated to be about 10 to 25 billion EUR per year. This means, that if consumers 
could have consumed the power that was not supplied, it would have brought benefits of 
10-25 billion EUR. It should be noted that the estimation of the value of lost load is 
extremely uncertain, and estimates in the literature vary substantially. 


1.3.2.1 Outages in power generation (Output 3) 


Power generation assets, such as thermal power plants, provide a certain generation 
capacity to the power market. However, situations are likely to arise where the capacity is 
not available e.g. in situations of technical malfunction or simply when the plant needs 
scheduled maintenance. Such periods of unavailability can be problematic for the power 
system, in particular when it is combined with other stressful events. The possibility of 
having an unplanned failure is a key reason for TSOs to regulate in the market e.g. by 
contracting reserve capacity. However, also scheduled downtime due to e.g. maintenance 
can be problematic to the system. While it does not in itself pose a direct problem, it 
reduces the overall capacity available in the system, and thereby increases the 
vulnerability of the system consequently increasing the likelihood of a trip in a different 
generator or interconnectors becoming a severe event. Most TSOs also regulate this by 
e.g. ordering in which time periods specific power plants are allowed to go out for 
maintenance. 


In 2016, approximately 886 GW of power generating capacity was available in the EU. 
However, due to outages this full capacity was not available for generation all the time. In 
fact, more than 14,700 outages have been reported in Europe each year (2015- 
2016 average), which corresponds to 17 outages each year per installed GW of 
generation.? 


When taking into account the length of the individual outages, we estimate that the total 
non-generated electricity due to outages in the EU in 2016 was about 13 TWh. 


It is a rough approximation and based on several simplifying assumptions. 

IEEE (2007) Blackout experiences and lessons, best practices for system dynamic performance, and the role 
of new technologies 

These numbers include only outages above 100 MW, Commission Regulation No. 543/2013. This excludes 
most renewable energy generation except for large onshore and offshore wind turbines. 
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This means electricity that could have been generated if the power plants had not been 
out for whatever the reason. This corresponds to the 886 GW of power generating 
capacity in Europe being completely out for around 15 hours in the whole year of 2016, or 
to 0.17% of Europe’s capacity being out constantly. This number is significantly lower 
than just a few years back, having been reduced to 13 TWh from about 41 TWh in 
2013, 47 TWh in 2014 and 35 TWh in 2015 (see figure). The decline is actually likely to 
be even higher than this, as data availability for 2013 and 2014 was significantly lower 
than for 2015 and 2016.’ 


Figure 1 : Total non-generated electricity per year in Europe 
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Note: Croatia, Cyprus and Slovenia did not report outages in any of the years. 2015 is the first full year where the reporting of 
outages to ENTSO-E is mandatory; the data for 2013 and 2014 is therefore incomplete in the sense that it only shows the 
outages that were reported voluntarily. The real total non-generated electricity in those years might be much higher, meaning 
that the decline over time to 2016 is much more significant than it seems in this figure. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform. 


Forced/unplanned outages constitute the majority of the non-generated 
electricity and they occur significantly more frequently. About 60% of all outages 
are forced events. The duration of the outage is significantly shorter when it is forced 
than when it is planned (about 9 hours on average compared to 34 hours for planned 
downtime). Moreover, forced outages typically happen for larger assets than 
planned outages. Indeed, the average affected capacity for a forced outage is about 
410 MW compared to about 260 MW for planned outages (see figure). This suggests that 
larger generation assets are more susceptible to forced outages than smaller ones. 


7 Planned outages are typically periods where power plants are out for maintenance. These outages could 


potentially be reduced in duration and frequency, but cannot be prevented, as maintenance is necessary for 
stability in operation and required by law. 
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Figure 2 : Characteristics of an average outage in Europe, planned vs forced, 2010-2016 
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Note: Croatia and Cyprus did not report any outages. Malta did not report whether its outages were planned or forced. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform and survey answers. 


Fossil-fuel assets have by far the longest duration for planned outages of about 
20-40 hours on average, compared to less than 1 hour for wind turbines, 5 hours for 
nuclear plants and about 7 hours for hydro plants. The comparably high figure for fossil 
fuel assets is to a large extent driven by single power plants - mainly coal fired plants - 
that were taken down for several months. Generally, when looking across technologies, it 
is however worth noting that the majority of reported planned outages is quite short: 
61% of all reported planned outages last shorter than an hour, and 41% are even shorter 
than 15 minutes, see also Figure 31. 


Fossil-fuel based generation assets seem on the other hand to bounce back from a forced 
outage as quickly as an average generation type taking about 2.8 hours, similar to hydro 
and nuclear. That is faster than biomass (8.4 hours) but slower than off- and onshore 
wind assets using 1.8 and 1.3 hours respectively. 


Offshore wind turbines are most prone to outages, with 331 outages per 
installed GW - of which most are planned. It is important to note that these are relative 
numbers (per installed GW); as fossil-fuel based generation units still dominate the total 
fleet of generation units, these units will have more outages in absolute terms. Only very 
few outages are reported for onshore wind, and none for solar energy, which is due to the 
fact that the units are below the threshold of reporting obligation of 100 MW.? 


8 Following Commission Regulation No. 543/2013. This threshold is shortly discussed in footnote 37 in the 
chapter on output 3. 


Figure 3 : Number and duration of planned and forced outages for different generation 
types, 2010-2016 
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Note: Croatia and Cyprus did not report any outages. Malta did not report whether its outages were planned or forced. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform and survey answers. 


Just as an outage at a generation unit, an unavailable interconnector will limit the power 
generation capacity available to a country or region. Interconnectors have been 
unavailable for roughly 87,000 hours every year (2010-2015 average), which 
means that 10 ‘average’ interconnectors are unavailable in Europe at all times on 
average. The main reasons for the unavailability are especially maintenance, 
repair and new constructions, while forced outages (due to overload, false operation, 
failure, outside impacts or very exceptional conditions) happen very rarely, see Figure 4. 
Outages due to works at the infrastructure (e.g. at overhead lines or underground cables) 
tend to take longer; the longest durations are due to new constructions, where an outage 
takes almost 10 days on average. Forced outages can on average be fixed much quicker, 
after 3 hours to 3 days. The somewhat increasing trend in the number of outages should 
be seen in light of the increasing number of interconnectors. 
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Figure 4: Total duration of interconnector outages, 2010-2015 
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Note: Incomplete data for 2016, therefore shown until 2015. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform. 


1.3.2.2 Disruptions of power supply to consumers (Output 4) 


Power supply is very reliable in the EU. For an average EU country, power consumers 
will on average have access to power in 99.948% of the time (measured in 2014). 
An average EU power consumer will face about 1-2 significant disruption events each 
year.” Since 2010 this number has fallen from 1.7 to 1.4 in 2014. The number of 
disruptions and the duration vary across individual Member States with Luxembourg, 
Denmark, Germany and the Netherlands at the high end of reliability with around 
20 minutes to half an hour of disruptions per customer per year, and Romania, Latvia 
and Croatia in the low end with about 550-850 minutes of disruptions per customer 
per year, see figure. 


“Longer than 3 minutes” is the most commonly used threshold across the EU, with 24 of the 28 Member 
States using it. The four Member States reporting disruptions in a different way are Denmark and the 
Netherlands, which report all disruptions of at least 1 minute and 5 seconds respectively, as well as Cyprus 
and Malta, which do not have a classification. 
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Figure 5 : Average minutes lost per year and customer due to significant disruptions 
(planned and forced), 2010-2014 
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Note: The graph shows a weighted indicator. The weighting is described in Table 4, and allows for an overall interpretation of 
the indicator as “per customer”. No data available for Belgium, Bulgaria, Cyprus. The average is a simple average across 
countries and is calculated based on the EU28-countries with available data. 


Source: Copenhagen Economics based on CEER data. 


When a power disruption happens, it is either due to a malfunction in the grid 
infrastructure at distribution level or transmission level, or due to a lack of generated 
power. While the vast majority of the disruptions occur due to problems in the 
grid such as weather-related breakdown of lines or transformer station malfunctions, a 
few incidents have been reported where lack of power generation to meet supply has led 
to disruptions. 


We have identified about 175 instances between 2010-2016 where a lack of power led to 
a supply disruption. The bulk of those incidents happened in the UK (100 instances since 
2010) and Malta (69 instances since 2010) in the early years of that 7-year-period. Both 
countries show a strong downward trend: in the recent years of 2015-16, only nine out of 
12,927 outages in the UK (0.07%) and five out of 192 outages in Malta (2.6%) led to 
disruptions. Ireland, Poland, and Romania report single cases as well, see Table 5 for 
details. Ireland for example has experienced a single case since 2010 where the tripping 
of a large generation unit has led to a supply disruption. In the same timeframe, Ireland 
reported more than 2,000 outages in total, meaning that most of them did not lead to a 
supply disruption. 13 countries stated that there has been no single case where an outage 
has led to a disruption since 2010 (see Table 5). Disruptions due to lack of power have 
therefore been very rare historically, and there is no evidence that the increased 
renewable energy deployment has led to more disruptions. Concerns remain going 
forward that unless additional measures such as e.g. changes in market design, 
intermittent sources could lead to more disruptions due to lack of power. +° 


10 See e.g. Statnett, Fingrid, Energinet and Svenska Kraftnät (2016), Challenges and Opportunities for the 


Nordic Power System, simulating amongst others that the capacity margin in Finland is likely to fall from 
1.400 MW to 90 MW in 2023 in a median year, and drop to negative 680 MW in a cold year once in 10 years. 
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When the disruptions are related to infrastructure, it is almost always caused in the 
distribution grid. All countries answering this particular question in our surveys stated 
that disruptions typically happen at distribution level. Failures in transmission 
infrastructure happen much more rarely, however when they do happen, the impacts 
are significantly more severe, sometimes leaving very large geographic regions 
without access to power. The exception to this is transmission lines between countries 
(interconnectors), where case examples show that interconnector failure alone very rarely 
gives rise to significant distress to the power system let alone actual disruptions to 
consumption. Instead, it leaves the national/regional power market more vulnerable to 
additional simultaneous failures in e.g. local generation assets (reduced capacity 
margins), and may increase the power prices in the countries. 


We find that about a third of the minutes of disruptions are planned disruptions 
e.g. for maintenance or construction purposes, and two thirds are unplanned 
disruptions. The primary reason for the unplanned disruptions are so called false 
operation, failure of equipment or material damage (40%) followed by severe weather 
conditions and natural hazards (30%). We find very little evidence that disruptions 
have been due to a malicious attack. 17 countries report that no disruption was 
caused by a malicious attack, only Poland, France and Italy report single cases of thefts of 
infrastructure elements that have caused disruptions. 


We estimate that for the EU in total approximately 600-850 GWh of electricity 
annually have not been supplied to consumers in the period 2010-2014. This 
constitutes both planned and unplanned disruptions. 


1.3.2.3 Use and potential of demand side flexibility to avoid disruptions (Output 5) 


Traditionally, balancing the power system to ensure the right frequency has been a 
question of regulating the supply side, i.e. ramping power generators up or down. 
However, the demand side is increasingly seen as a potential resource for balancing 
purposes due to several reasons. Firstly due to technological developments making 
demand response easier to use for balancing purposes, and secondly due to the expected 
increase in large-scale demand assets in individual households such as electrical vehicles 
and heat pumps. 


Demand side flexibility is viewed upon with great potential as it is indeed the 
current inflexibility that gives rise to the discussions about reserve availability, capacity 
payments etc. If it could be ensured that consumers were easily and cheaply available to 
communicate and respond flexibly to price variation, most of the issues related to energy 
system balancing would be solved. 


However, there are still a number of challenges left with respect to large-scale 
utilisation of the demand side as a balancing asset such as 1) it is not deemed as 
reliable as supply assets, 2) it cannot typically be available for as long as supply assets, 
3) the demand assets are substantially smaller in size and therefore requires significant 
coordination efforts to mobilise sizeable capacity, 4) it is geographically less predictable 
and may be ‘locked into’ local bottlenecks, and 5) it is relatively expensive to mobilise 
especially household assets as it requires investments in e.g. smart metering equipment, 
new business models and new regulation and market design models, including e.g. 
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participation requirements in the balancing market, such as the minimum bid size, which 
differs across Member States. ! 


Several of these challenges are likely to be overcome e.g. through better coordination 
and information flow between TSOs on the one hand and DSOs and balance responsible 
parties on the other hand.!? What remains to be seen is to what extent the demand side 
can act as a reliable source of adequacy instead of e.g. a 200-400 MW power generation 
plant, which will require significant efforts in terms of e.g. aggregation. A more immediate 
role for the demand side could be to perform services closer to real-time operation, e.g. 
in fast-responding automatic (frequency controlled) reserves, where requirements on 
capacity size and availability periods are lower. 


It has generally been very difficult to find information of the extent to which demand side 
assets have contributed as an effective tool to avoid brownouts in actual periods of stress 
to the system, as this is rarely recorded and/or published by TSOs. To the extent that 
demand side assets contribute to balancing mechanisms and reserve pools, we expect 
that these have been called upon in times of severe distress 


Some countries engage in specific demand curtailment programmes used in 
emergency circumstances. These programmes allow for a more controlled 
disconnection of consumers before an actual brownout that disconnects an entire 
geographic area. 12 countries seem to have such voluntary demand curtailment 
mechanisms in place and in about 8 out of these 12, we have found evidence that 
these demand side programmes have been activated, see map. 


11 See for example the study “Unlocking flexibility - Nordic TSO discussion paper on third-party aggregators” 


by Energinet, Fingrid, Statnett and Svenska Kraftnat 
E.g. to prevent that a TSO-activation of a local flexibility asset does not lead the local balance responsible 
party to counteract this to achieve local balance. 
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Figure 6 : Activation of voluntary demand curtailment mechanisms 2010-2016 


w 


Wl Evidence of activation © À 


lll. No evidence of activation 


1.3.2.4 The socio-economic costs of disruptions (Output 6) 


Disruptions impose a welfare loss to society, as consumers willing to pay the price for 
electricity do not have the opportunity to consume it. That limits the production of 
commercial consumers and the value of leisure of private consumers, and can in some 
instances lead to damages of products or machinery. The value of lost load (VoLL) is an 
estimate for those costs. 


The VoLL depends on a wide range of factors. It varies for example depending on the 
time of day, season and duration of the disruption, as well as with the individual 
preferences and production functions of the private and commercial consumers affected. 
The VoLL is moreover typically higher for countries with a high-income level, higher for 
commercial than for private consumers, and higher for forced than for planned 
disruptions. 


In order to estimate the value placed by consumers on electricity, all the above 
differences should optimally be taken into account. However, current estimates of VoLL 
show tremendous variation seemingly driven by differences in methodological approach 
and data availability and to a lesser extent of the underlying economically driven 
differences in VoLL. All estimates, and EU averages in particular, should therefore be 
considered a rough approximation rather than the “true” value. 
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Based on an assessment of existing estimates combined with an extrapolation, we find 
that the household-VoLL is about 5-10 EUR/kWh at the European average, meaning 
that a one kWh of non-supplied electricity to an average household in Europe implies 
costs to that consumer of 5-10 EUR. For non-households like industrial or 
commercial consumers, we find an average VoLL of 20-30 EUR/kWh. 


Those estimates show that the disruptions in Europe in the period 2010 to 2014 gave rise 
to a loss to consumers of approximately 10 to 25 billion EUR annually.” 


13 Not considering potential savings in generation cost from not generating the electricity. 
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2. INTRODUCTION 


The « Study on the quality of electricity market data of transmission system operators, 
electricity supply disruptions, and their impact on the European electricity markets” was 
launched by DG ENER under the framework contract ENER/A4/516/2014. The contract 
was assigned to the consortium composed of VVA (lead), Copenhagen Economics, Neon 
and Deloitte. 


The contract was signed on 3% May 2017 and the study implementation period is 7 
months. 


2.1 Objectives and scope of the study 


The study aimed at filling the existing data gaps on supply disruptions of electricity due to 
outages on production facilities, transmission or other grid assets. 


The study consisted in the collection, completion, calculation, presentation and analysis of 
data on the current reporting practices of electricity transmission operators, on the 
existing public European electricity market databases, on the quality of the available 
market data and on the costs of electricity supply disruptions for customers. The study 
covered the 28 Member States and the period 2010-2016. 


Based on this extensive data collection, the study provides an in-depth analysis of 
reasons behind and the nature of electricity supply disruptions, the cross-border 
implications, the impact of voluntary demand curtailments, as well as the costs and losses 
incurred and how they impact market prices. 


Having a better understanding of the causes of disruptions of electricity and their impact 
on the European market is necessary to contribute to enhancing the electricity security of 
supply in the EU and to provide updated and evidence-based energy policies. 


2.1 Overall approach 


The study consisted of two work packages, both covering the 28 EU member states and 
involving data-intensive analyses, but independent from each other otherwise. The table 
below summarises the overall methodological approach of the study. 


Table 1 : Overall methodological approach 
Work Packages | Outputs 

1) Work Package 1: Evaluation of TSO | Output 1: Analysis of ENTSO-E datasets 

and ENTSO-E data 
Data collection and analysis on the 
completeness, accuracy, timeliness of the 
datasets, accessibility and friendliness of TSO 
and ENTSO-E database. 


Output 2: Terms of data management and 
access 


Analysis of legal data provision terms of TSO 
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Work Packages 


Outputs 
and ENTSO-E to check if compatible with the 
Regulation 543/2013 on submission and 
publication of data in electricity markets. 


2) Work Package 2: Analysis of supply 
interruptions 


Output 3: The extent and source of 
electricity outages 


Information on electricity outage data in 28 EU 
Member States on a monthly basis and by 
generation technology, including on plants 
(generation units), the nature of outages 
(planned or unplanned and the cause), their 
length in time, the reduction of the available 
capacity and the amount of non-generated 
electricity. 


Output 4: Understanding electricity 
supply disruptions events 


information on the extent of electricity supply 
disruptions in the EU 28, and the reasons for 
these disruptions. Definition of a typology for 
significant supply disruption events and 
identification of the features of these events 
(cause, length, reduction of available capacity, 
amount of non-supplied electricity). 


Output 5: The use of voluntary demand 
curtailment 


Analysis of the impact of significant events on 
voluntary demand curtailments and how those 
contributed to minimilazing the impact of 
elecitrity supply disruptions. 


Output 6: Value of lost load 


Value of lost load (VoLL) resulting from the 
significant electricity supply disruption events, 
in monetary units and across the EU Member 
States. 


3) Final report : Conclusions and 
recommandations 


Output 7: Final report 


Presentation of the final results of the study, 
definition of concepts, presentation of the 
nature of data collections, of the limits of the 
applied methodology, of the solutions adopted 
to deal with missing or inconsistent data. 
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2.2 Structure of the report 
The structure of the remaining inception report is as follows: 


e Chapter 3: Findings for Work Package 1 
e Chapter 4: Findings for Work Package 2 
e Chapter 5: Annexes 


Annexes: 


e Annex 1 - Content of Commission Regulation (EU) 543/2013 

e Annex 2 - Overview of data items available on ENTSO-E TP 

e Annex 3 - Questionnaire for online user survey (Output 1) 

e Annex 4 - Survey questionnaire for Output 3 (with electricity producers and TSOs) 
e Annex 5 - Survey questionnaire for Output 4 (with national regulators and TSOs) 

e Annex 6 - Survey questionnaire for Output 5 (with TSOs) 

e Annex 7 - List of stakeholders 

e Annex 8 - List of literature 
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3. FINDINGS FOR WORK PACKAGE 1 


Work Package 1 is an evaluation of the ENTSO-E Transparency Platform and consists of 
two outputs: Output 1 is an analysis of the data quality and user friendliness of the 
platform by means of a data analysis and a survey among platform users. Output 2 is an 
analysis of the legal accessibility and terms of use of the platform. The goal of this Work 
Package is to be constructive and solution-oriented by providing insights that contribute 
to improving the platform. Neon was responsible for this work package. 


3.1 Output 1: Analysis of ENTSO-E datasets 


3.1.1 Objective and approach 


The objective of this Output is to evaluate the quality of data published through the 
ENTSO-E Transparency Platform (TP) as well as the user friendliness of the platform itself. 


To deliver this assessment, we applied four complementary approaches. First, we 
reviewed the other previous TP assessments (e.g. ENTSO-E internal quality assessments, 
ETUG reports, ACER opinions). Then, we carried out a new statistical data analysis of the 
data for all EU Member States in the period 2015-2016. We carried out an online survey 
with data users, and received 80 answers. Finally, we conducted semi-structured 
interviews with experts reflecting the variety of types of users, including market 
participants, consultants, NGOs, data service providers and researchers, as well as 
national authorities and EU institutions. 


3.1.2 Findings 


All reported findings are based on observations and user input obtained between April and 
October 2017. Since the TP architecture allows updates and improvements for historical 
data, data quality may have increased since our analysis. Data items are referred to by 
the name listed on the TP as well as the corresponding article number from Regulation 
543/2013. 


3.1.2.1 Completeness 


Assessing completeness means verifying whether all data items specified in Regulation 
543/2013 are available on the TP for all geographic entities that apply and for all time 
steps since January 5, 2015, when the Regulation came into force. We identified two 
types of issues regarding completeness: 


e Missing data: data that should be published are not (data gaps) 
e Information about missing data: users are not informed about data gaps 


Additionally, some users requested broader coverage and identified data they would be 
interested in that go beyond what is prescribed in Regulation 543/2013. 


It should be noted that many data items are useful only if they are reported completely. 
Missing load, price, transmission or generation data even in “only” 1% of all time steps 
can render the raw time series useless for many use cases. For such time series, anything 
below 100% completeness would seem unsatisfactory. Even though gaps can be filled 
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using data processing software, doing so could introduce a bias to the data because the 
gaps might not be random. 


(a) Missing data (data gaps) 


As visible from Figure 7, online survey respondents reported missing data in every data 
domain. Users’ perceptions of missing data were measured by asking the question “Are 
there missing observations or gaps in the data?” They were then given the option to 
answer “There are many gaps”, “There are some gaps”, “There are no gaps” or “I’m not 
sure”. Since users were not given any instructions or methodology for defining how many 
gaps constituted each category or how to define a gap, the survey did not measure the 
objective completeness of the data. It should further be noted that users are not always 
informed about updates to data after they last worked with them. However, the survey 
shows user perceptions of missing data on the TP. According to the results, users 
perceived Outages as the data domain with the most gaps: not a single respondent did 
not report missing data and more than 40% find “many gaps” in the data. In the 
following, we will discuss more detailed analyses of items of the data domains Load, 
Generation, Transmission and Balancing. 


Figure 7. Percentage of users who noticed gaps in different data domains. 
Key point: For every data domain, at least 80% of users noticed data gaps. 
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Notes: Data domain names with asterisks represent those for which fewer than 30 users responded. 


Load 


Within the data domain Load, we focused on the data item “Actual Total Load” (6.1.A). 
Figure 8 shows the availability of “Actual Total Load” (6.1.A) by country. The pattern of 
data unavailability suggests different reasons for missing data: for some countries, data 
came in late at first, but are nearly complete since then (Cyprus, Denmark, Sweden). 
Other countries have many short gaps (Romania, Slovakia, Spain). Yet other countries 
feature a larger number of longer gaps that seem to be randomly distributed over the 
time period (Ireland, Italy, Lithuania). A positive observation is that the “extra hour” in 
March due to daylight savings time—a notorious weak spot of power system data—does 
not seem to pose a systematic problem in the “Actual Total Load” (6.1.A) data. 


Figure 8. Completeness of “Actual Total Load” (6.1.A) by country. 
Key point: The patterns of missing data are different from country to country. 
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Notes: The figure shows data availability in hourly resolution. Very short gaps might not be visible. For higher resolution see 
https://neon-energie.de/transparency-platform. 


Generation 


We studied the items “Aggregated Generation per Type” (16.1.B&C) and "Actual 
Generation per Generation Unit" (16.1.A). Figure 9 shows the former, focusing on the 
most common technologies. Coloured cells show the share of observations missing 
(reported as "N/A" on the TP website). White fields containing "n/e" indicate that 
generation data from that country and technology are not expected on the TP. Cases of 
100% "N/A" also could indicate a misconfiguration where no data are expected; that is, 
"n/e" should be reported instead. Croatia (all values "N/A") as well as Luxembourg and 
Malta (all values "n/e") do not report any data for this data item. 


Few time series are complete. Coverage is nearly complete in Austria, Belgium, Germany, 
Denmark and Portugal. In Italy and Slovenia, a year is missing for some or all 
technologies, resulting in shares of around 50% missing values for the two years covered. 
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Figure 9. Completeness of “Aggregated Generation per Type” (16.1.B&C) by country. 
Key point: For the majority of countries, a significant amount of generation data is missing. 
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Notes: Share of missing values (reported on TP as “N/A”) for selected technologies. Due to space constraints, we have 
restricted the figure to a subset of all technologies. Latvia operates one hydropower plant that was classified as “Hydro Water 
Reservoir” until 25.03.2015 and as “Hydro Run-of-river and poundage” afterwards, leading to two columns where one of two 
values is always “N/A” or “n/e”. 


Figure 10 shows how the completeness of “Aggregated Generation per Type” (16.1.B&C) 
evolved over time. It shows the number of observations per week aggregated over all 
countries and production types and compares this to the expected total if all data were 
reported. Under the assumption that no country has decommissioned all plants of a 
certain type, the number of expected observations does not change over time. The 
number of actual observations per week seems to increase from 2015 to 2016, indicating 
improved completeness. However, this pattern is due to the appearance of Italian data in 
2016, which were missing in 2015 altogether. Disregarding the Italian data, the overall 
completeness of the data shows some ups and downs, but seems to stabilize around 1000 
missing observations per week. 
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Figure 10. Weekly number of “Aggregated Generation per Type” (16.1.B&C) 
observations. 
Key point: We cannot identify a trend toward improvement of completeness over time. 
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Notes: In a week, the expected number of observations 42,504 = 168 hours x 253 country-type combinations. Excepting 
Italy, it is 40,320 = 168 hours x 240 country-type combinations. The total number of country-type combinations on the FTP 
server is 260; however, this includes combinations that are always marked as “n/e” on the TP website and are thus 
disregarded. 


We assessed the data item “Actual Generation per Generation Unit” (16.1.A) for the year 
2016. On average, 5% of observations are missing. Less than half of all units report data 
without gaps. Figure 11 displays the generation units that reported the most missing data 
in 2016. Some units provided hardly any data; there are more than 100 units for which at 
least 40% of all observations are missing. Most of them are situated in the United 
Kingdom, suggesting a systematic problem with the reporting there. 
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Figure 11. Missing “Actual Generation per Generation Unit” (16.1.A) in 2016, averaged 
by country. 
Key point: In Ireland, the United Kingdom and Bulgaria, generators on average provided less than 
60% of all data. 
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Source: Own figure based on data provided by Dave Jones, Sandbag. 


Notes: Figure shows the share of missing observations by country for EU Member States. There are no data for Irish generation 
units after 24.05.2016 and none for Bulgarian units before 16.05.2016, leading to high percentages of missing values. In the 
United Kingdom, generators on average provided little more than half of all data. No data at all are provided for Cyprus, 
Luxembourg, Malta and Croatia, possibly due to no generators »100 MW existing in these countries. Generation units located in 
Sweden, Hungary, Poland, Romania, Latvia, Slovakia and Slovenia provided virtually all («0.596 missing) data. 


Responses by interviewees and survey participants are consistent with our findings. In 
addition, it was pointed out that the 100 MW reporting threshold for individual units 
seems to be applied inconsistently—sometimes to entire power stations, in other cases to 
individual electricity generators. More issues with completeness included reporting gaps in 
German gas plants, Spanish solar production and for all production types in the 
Netherlands. 


Transmission 

Within the data domain Transmission, we evaluated the data items "Day-ahead Prices" 
(12.1.D) and "Scheduled Commercial Exchanges" (12.1.F). "Scheduled Commercial 
Exchanges" (12.1.F) is one of the patchier data items, as Figure 12 illustrates. For some 
time series, a year of data is missing, which is the case for some of the Italian, Lithuanian 
and Norwegian borders. Others exhibit frequent shorter gaps, e.g. borders between 
Bulgaria and Greece and their respective neighbours. 
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Figure 12. Completeness of “Scheduled Commercial Exchanges” (12.1.F) by bidding zone 


borders. 


Key point: Exchange data are patchy with different patterns of missing data. 


# # 224 


2015-01 © 

2015-02 
2015-03 
2015-04 
2015-05 
2015-06 
2015-07 
2015-08 
2015-09 
2015-10 
2015-11 
2015-12 
2016-01 
2016-02 


2016-03 
2016-04 
2016-05 
2016-06 
2016-07 
2016-08 
2016-09 
2016-10 
2016-11 
2016-12 


Notes: The figure shows data availability in hourly resolution. Very short gaps might not be visible. For higher resolution see 
https://neon-energie.de/transparency-platform. 


As visible in Figure 13, "Day-ahead Prices” (12.1.D) show fewer gaps than the other data 
items; however, there is only one complete time series of day-ahead prices (Spain). Italy 
alone is made up of 18 bidding zones, most of which have a period of missing data in 
October 2016. No price data are expected for bidding zones that have not introduced a 
power exchange. This was the case in Bulgaria (ESO BZ) and Croatia (HOPS BZ) prior to 
January/February 2016 and still is the case in Malta. However, Bulgaria (ESO BZ) reports 
prices only from November 2016 and Croatia not at all. Overall, there is no general trend 
of improvement over time. 
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Figure 13. Completeness of “Day-ahead Prices” (12.1.D) by bidding zone. 
Key point: For almost all bidding zones, day-ahead prices are incomplete. 
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Notes: On average, 5% of observations are missing, with some gaps in almost all bidding zones. Until March 2017, price data 
for Poland (PSE SA BZ) were not expected for hours with zero energy exchange with neighbouring countries, which was the 
case 25% of the time in 2015-2016. The figure shows data availability in hourly resolution. Very short gaps might not be 
visible. For higher resolution see https://neon-energie.de/transparency-platform. 


Responses by interviewees and survey participants are consistent with our findings. Noted 
issues included incomplete "Physical Flows" (12.1.G) and "Scheduled Commercial 
Exchanges" (12.1.F). 


Balancing 


From the Balancing data domain, the data item "Total Imbalance Volumes" (17.1.H), 
which is reported per market balance area, was chosen for analysis (Figure 14). The 
Finnish TSO Fingrid provides data from March 2015 onwards and has frequent gaps. 
Overall, however, completeness is better than in any other data item we assessed: for 
two-thirds of all balancing areas, fewer than 0.2% of all observations are missing. About 
one-quarter of all imbalance volume time series are complete. 
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Figure 14. Completeness of “Total Imbalance Volumes” (17.1.H) by market balance area. 
Key point: For most balancing areas, imbalance volumes are nearly complete. 
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Notes: Data from Lithuanian TSO Litgrid are uneven between June 2015 and April 2016 in the CSV files retreived from the FTP 
server but complete when accessed through the GUI. The figure shows data availability in hourly resolution. Very short gaps 
might not be visible. For higher resolution see https://neon-energie.de/transparency-platform. 


Responses by interviewees and survey participants are consistent with our findings but 
also point out issues with other items in the data domain, including incomplete "Amount 
of Balancing Reserves Under Contract" (17.1.B). One user familiar with the balancing 
working group found it problematic that it is comprised of TSO users with no market 
participants represented. 


(b) Reporting data gaps and public documentation of issues 


Users not only have raised concerns about incompleteness but also emphasized that there 
is no information available about the status and degree of completeness and no warnings 
about incomplete data. This forces each user to monitor completeness individually. 


When users encounter gaps, there is no process to publicly flag missing information as a 
warning for other users. There is also no direct way of contacting Data Providers or 
Primary Data Owners. The only way to inform the Data Providers of gaps is through the 
ENTSO-E service desk. However, there is also no public record of service desk inquiries or 
issues. As a result, TP users waste resources trying to determine whether data are 
sufficiently complete for analyses. This was one of the most persistent complaints we 
received from users; it is also noted in an ETUG summary of user feedback. 


Additionally, some users have suggested that the reason for the missing data should be 
published to facilitate its correction; for example, if it is because a TSO has not submitted 
the data, the TP user could call the TSO directly rather than being routed through the TP 
service desk, which likewise must answer redundant calls. Publishing such data could help 
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create accountability for those institutions that have a history of failing to completely 
report data. 


(c) Broadening the scope 


Some stakeholders have an interest in additional data that could be available on the TP. 
This is different from the missing data reported above because these suggestions go 
beyond what Regulation 543/2013 prescribes to be published. While these are therefore 
not issues of incompleteness with respect to the Regulation, we believe reporting user 
needs and preferences is valuable and gives an impression of what users would consider 
to be a complete database. 


The most common requests include a variety of price data, renewable forecasts published 
earlier and for longer timeframes, net transfer capacity values and detailed generation 
per unit for plants below 100 MW. 


However, most users preferred to focus on improving the quality of the existing data 
items rather than adding any more items at this stage. 


3.1.2.2 Accuracy 


The accuracy analysis aims to identify whether data are “correct”. We compared values 
on the TP to values reported elsewhere. It should be noted that differing values may 
result from differing data definitions rather than being proof of inaccurate values; 
however, our results concern cases in which the data were reasonably comparable. We 
found four major issues related to the accuracy of the TP: 


e Inconsistencies with other ENTSO-E data, 

e Inconsistencies with other data sources, 

e Information about inaccuracy: users are not informed about incorrect data and 
e Inaccurate data definitions. 


From January to June 2017, there were 68 service desk requests regarding discrepancies, 
differing values and incorrect data. In the ETUG survey, 18% of users characterised their 
trust in the reliability of TP data as “little”, with another 41% responding “a moderate 
amount”; more than 54% of users had noticed data inconsistencies while using the TP. In 
its opinion on the first update of the MoP, ACER noted that despite improving other 
aspects of the TP, ENTSO-E had failed to address improvements in assuring data quality. 


In our own Survey, half of all respondents reported TP data to be inconsistent with other 
sources, mentioning the ENTSO-E Ten-Year Network Development Plan, Yearly Statistics 
and Mid-term Adequacy Forecast; Eurostat; SKM SYSPOWER; balancing data provided by 
RTE; installed capacity per production unit published on regional REMIT platforms; 
planned production published on EEX; sum of generation from BDEW and national TSO 
and industry reports. According to our online survey, users noticed at least some 
inaccuracies in all data domains, as displayed in Figure 15. Users were asked the question 
"Do you find data on the platform to be accurate (correct)?". They were then given the 
options “Most values seem implausible”, “Some valuables seem implausible”, “Data 
seems correct” and “I’m not sure”. Since users were not given instructions or 
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methodology for defining how many implausible values constituted each category, the 
survey did not measure the objective accuracy of the data. It further should be noted that 
users are not always informed about rectifications of the data after they last worked with 
them. However, the survey results show users’ perceptions of accuracy on the TP. 


Figure 15. Percentage of users who noticed implausible values in different data domains. 
Key point: For all data domains, about half or more of users have noticed implausible values. 


0% 25% 50% 75% 100% 


Load 


Generation E Data seems 
correct 
[Some 
implausible 
values 
Congestion . implausible 
Management 


Notes: Data domain names with asterisks represent those for which fewer than 30 users responded. 


(a) Comparison with other sources 


One way to check whether TP data are accurate is to compare them to other trusted 
sources. However, for many data items other sources do not exist, are blocked by a 
paywall, are proprietary or are not available in one central spot. We therefore focused on 
a few data items for which we compared TP data with sources such as ENTSO-E's Power 
Statistics (formerly "Data Portal"), Eurostat and data collected from individual TSOs' 
websites. For some of these data items, it is possible that the definitions differ depending 
on the source; however, we believe the results give a valid analysis of data 
inconsistencies. 


Load 


We compare the data item "Actual Total Load" (6.1.A) to two other sources of load data: 
ENTSO-E provides load data in sections of its website separate from the TP called "Data 
Portal" (for data from years until 2015) and "Power Statistics" (years after 2016). It is our 
understanding that the data provided by the Data Portal and the Power Statistics are 
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sourced and processed separately. Monthly aggregated load data are available under the 
titles "Monthly consumption”! (Data Portal) and "Monthly Domestic Values"? (Power 
Statistics). A second source of load data is Eurostat’s “Supply of electricity - monthly data 
(nrg 105m)"Ÿ, These sources differ in two important aspects: 


e TP data are delivered close to real time (one hour after the operating period), 
while the other sources undergo revisions. 

e TP data require total load, while the Data Portal/Power Statistics may report a 
share of the total, as indicated by the possibility to report a country-specific 
“Representativity Factor”. 


The first difference implies that we can expect random deviations between TP and the 
other sources resulting from close-to-real-time estimation errors. These errors should not 
be systematic, i.e. they should average out over longer time periods. The second 
difference implies that the Data Portal/Power Statistics data and Eurostat data in those 
countries that have a Representativity Factor smaller than 100% could be smaller than TP 
values. The reported Representativity Factors!’ are always 100% for all countries, 
however; thus, these sources should be reporting total load. 


In almost all countries we find significant, persistent deviations among all three sources; 
in most cases, TP numbers are smaller than the other statistics (Figure 16). Deviations in 
the double-digit percentage range are not uncommon. Moreover, deviations vary among 
countries: in Slovakia, TP load is somewhat larger than both other sources while in 
Austria, it is about 20% smaller.‘® This suggests that the deviations are not due to a 
difference in data definition among sources. 


^ https://www.entsoe.eu/data/data-portal/consumption/Pages/default.aspx 


https://www.entsoe.eu/data/statistics/Pages/default.aspx 
http://appsso.eurostat.ec.europa.eu/nui/show.do?dataset-nrg. 105m. Eurostat does not report electricity 
consumption explicitly: as we learned upon enquiring with their help desk, Eurostat explained that in their 
Electricity the data item "Gross inland consumption" should in fact be read as net imports. Assuming that 
electricity consumption equals gross generation + imports, we thus calculate electricity consumption by 
adding "Gross electricity generation - Total" + "Gross inland consumption". 
https://www.entsoe.eu/data/statistics/Pages/monthly domestic values.aspx 

For the Austrian case, we were informed by ENTSO-E that "the reason for the deviation in Austria results 
from different definitions of the respective sources. On the Transparency Platform, Total Load includes only 
data of the control area APG. Instead, the values on Power Statistics include data for the whole country (also 
including data of large industry with own production units and railroad consumption, which are not directly 
connected to the grid of APG)." 
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Figure 16. Deviation of load between TP and other sources. 
Key point: “Actual Total Load” (6.1.A) values are inconsistent with other sources’ load data, 
including ENTSO-E Power Statistics. The deviations are often significant in size (>10%). 
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Notes: 2015-16. Blue bars are calculated as “Eurostat minus TP” and orange bars as “Data Portal/Power Statistics minus TP”. 


Generation 


We compared the data item “Aggregated Generation per Type” (16.1.B&C) to other 
sources of generation data: for Germany, we compared all technologies to data published 
by two official German sources. To get a comparable dataset for yearly generation, net 
generation is taken from the German Federal Statistical Office (Destatis)!?. Generation 
data by wind and solar and most biomass units are missing in this dataset since it only 
includes generation by units with an installed capacity >= 1 MW”. For these production 
types, gross data, which includes consumption by power plants, are taken from AG 
Energiebilanzen?!, Since differences between net and gross generation are minor for 
renewables, this is deemed a reasonable approach. 


Figure 17 shows the result of the comparison. We find differences between the two 
datasets for most production types. Differences for individual production types could be 
due to diverging rules on assigning individual power plants to production types but should 
cancel each other out when aggregating all production types. This is partly the case, the 
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https://www- 
genesis.destatis.de/genesis/online/data?operation=abruftabelleAbrufen&selectionname=43311-0001 

2 https://www.destatis.de/DE/Publikationen/Qualitaetsberichte/Energie/ 
MBElektrizitaetsWaermeerzeugungStromerzeugungsanl066K.pdf?_blob=publicationFile#page=4 
http://www.ag- 

energiebilanzen.de/index.php?article id229&fileName-20170811 brd stromerzeugung1990-2016.xlsx 
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most significant example being fossil gas: on the TP 15 TWh are reported for fossil gas— 
67% less than the 47 TWh reported by Destatis. This is counteracted by a larger value for 
“Other” generation on the TP (39 TWh compared to 1 TWh on Destatis). Combined cycle 
gas turbines are reported as “Other” generation on the TP, explaining this discrepancy. 
For hard coal, the TP reports 78 TWh compared to 100 TWh on Destatis. Differences for 
renewable and nuclear generation are minor, with practically identical reported generation 
for wind and solar. When summing up all technologies, however, total generation as 
reported by Destatis is higher by 29 TWh than the TP data (551 TWh - 522 TWh), the 
reason for which is unclear. 


Figure 17. Comparing “Aggregated Generation per Type” (16.1.B&C) with 2016 German 
generation data from Destatis and AG Energiebilanzen. 
Key point: TP reports smaller values for fossil gas and hard coal compared to other sources. 
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For some countries, we collected hourly resolution wind and solar generation data from 
the websites of their respective TSOs?. As TSOs also submit those data to the TP under 


22 http://open-power-system-data.org/data-sources#8_Wind_and_solar_power_time_series 
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“Aggregated Generation per Type” (16.1.B&C), we expect that both sources should be 
identical. Figure 18 shows that this is the case for several countries, notably France and 
Austria. Additionally, solar data from Germany as aggregated from four individual TSO 
websites are almost always identical to the corresponding TP data (16.1.C) for Germany. 
However, for other countries, the two respective sources deviate regularly. Czech, Danish 
and Polish wind generation data (16.1.C) reported on the websites of the respective 
national TSOs often report different values than on the TP. Danish onshore wind 
generation (16.1.C) is practically never identical. This inconsistency is not due to different 
coverage, as for all countries in which deviations occur, they are sometimes positive and 
sometimes negative. In the case of Poland, according to ENTSO-E, differences can be 
explained by different calculation methods: hourly wind generation on PSE’s website is 
calculated as the average of quarter-hourly observations, while on the TP, hourly 
averages are based on more frequent observations. The TP values can thus be regarded 
as more accurate. 


Figure 18. Frequency of deviations of selected generation data between TP and TSOs. 
Key point: Some TSOs publish identical data on their websites and on the TP; others do not. 
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Notes: All countries for which we have collected data are listed. The selection was made based on availability and user 
friendliness of TSO data. Sometimes wind generation for one country is reported with up to two decimals precision in one 

source but as integers in the other. In order not to count this as a deviation, differences up to 1 MW are regarded as identical. 
Figure 19 compares TP wind generation data (16.1.C) with Eurostat’s “nrg 105m"? 
statistics as well as ENTSO-E's "Detailed monthly production”** from the Data Portal 


23 http://appsso.eurostat.ec.europa.eu/nui/show.do?dataset-nrg 105m 
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(through 2015) and "Monthly Domestic Values”? from the Power Statistics (from 2016 
onwards) for those countries in which the data are complete enough on the TP to allow 
for a comparison. In all countries, we find inconsistencies; however, some cases are less 
worrisome than others. France sticks out as a positive example and the United Kingdom 
(GB)? as a negative. During interviews, experts reported that GB data are problematic 
because offshore and/or plants connected at the distribution level are excluded from 
certain statistics, but we could not find any written documentation of this discrepancy. 


Figure 19. Deviation of wind generation between TP and other sources. 
Key point: TP wind generation data (16.1.C) often deviate significantly (21096) from other sources, 
including ENTSO-E Power Statistics. 
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(b) Other accuracy issues 


In some cases, inconsistencies can be identified by comparing with the TSO data or with 
data from a vendor. Both the ETUG summary of user feedback and many of the experts 
we interviewed identified national TSO data as often conflicting with values reported on 
the TP. However, in other cases, the only source for the data is ENTSO-E, and while data 
users may be sceptical of data accuracy they also cannot verify values. 


In our interviews and user survey, users mentioned issues with data in the Generation 
and Load data domains. Reported inaccuracies included values for “Aggregated 
Generation per Type” (16.1.B&C), especially in the Netherlands, and out-of-service 


^ https://www.entsoe.eu/data/data-portal/production/Pages/default.aspx 

25 https://www.entsoe.eu/data/statistics/Pages/default.aspx 

26 "GB" (not "UK") is the official ISO 3166-1 alpha-2 country code for the United Kingdom of Great Britain and 
Northern Ireland. Great Britain (excluding Northern Ireland) has the ISO 3166-2 code "GB-GBN". 
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production units in “Installed generation capacity per unit” (14.1.B) not labelled as such. 
Other issues included implausible generation values, including solar production (16.1.C) 
at night and values reported in “Day-ahead Generation Forecasts for Wind and Solar” 
(14.1.D) identical to those reported in “Aggregated Generation per Type” (16.1.B&C) for 
wind and solar. Users also expect the sum of generation (16.1.A/16.1.B&C) to correspond 
with the sum of “Actual Total Load” (6.1.A) and “Scheduled Commercial Exchanges” 
(12.1.F), which does not seem to be the case. 


(c) Reporting inaccuracies and public documentation of issues 


As in the case of completeness, users are not informed about inaccuracies, nor can they 
inform other users if they identify problematic data. Additionally, there is no established 
procedure for addressing inconsistencies between the MoP and the TP website. All 
requests are routed through the service desk and are not published. 


(d) Inaccuracy in data definitions 


Regulation 543/2013 specifies data definitions, but at a relatively high level. More details 
are given in the Detailed Data Descriptions, which are an annex to the MoP. However, 
these leave room for interpretation. Additionally, since they are an annex to the MoP, any 
changes to the data definitions require the same (lengthy) procedure as changing the 
MoP, even if they only serve as clarification. 


These unclear data definitions lead to inaccurate data with subsequent confusion on the 
user side along with inconsistent interpretation by Data Providers. Users we contacted 
had issues with definitions, including of forecast net transfer capacity, a lack of 
standardization among TSOs of data definitions and opaque measurement and estimation 
methodologies. One example is the case of "Aggregated Generation per Type" 
(16.1.B&C), in which generation by combined cycle natural gas power plants is not 
reported under "Fossil Gas" but under "Other". However, users only learned about this 
after inquiring at the service desk?’ as it is not mentioned in the documentation files. 


3.1.2.3 Timeliness 


Assessing timeliness means confirming that data are published on the TP within 
reasonable timeframes, ideally those specified in Regulation 543/2013. We relied on user 
input for the timeliness assessment, in particular from market participants. We 
summarize the issues with timeliness as follows: 


e Outage data and UMMs 
e Overwriting forecasts 
e Delays in data availability 


From an email from the ENTSO-E service desk, Jul 4, 15:02 CEST: “Other means other conventional and in 
our case it includes combined cycle gas turbines. Definition: Others = Total generation - (a+b+c) 

a) Renewable energy (based on projections and forcasts [sic]). 

b) Power Stations directly connected to the High-Voltage Grid (Measurements). 

c) Fossil Coal-derived gas (Schedule based, no measures)” 
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Often, there is a trade-off between timeliness and accuracy: publication of data shortly 
after real time often requires relying on estimation and extrapolation. Later, more 
accurate measurements become available. In the energy sector, it is not uncommon for 
some statistics to be revised during a period of up to three years. 


Issues with timeliness are most relevant for users working on a close-to-real-time basis; 
that is, mostly market participants. Figure 20 shows the results of the online survey 
regarding user perceptions of whether data were published in an acceptable timeframe. 
Users were asked the question “Do you find data on the platform to be available when 
you need it?”. They were then able to answer “Data is rarely available when I need it”, 
"Data is usually available when I need it", "Data is always available when I need it" or 
"I'm not sure". Since users were not given any instructions or methodology for defining 
how many untimely data values constituted each category, the survey did not measure 
the objective timeliness of the data. It should further be noted that users are not always 
informed about accelerations of the data publication process after they last worked with 
the given data. However, the survey results show users' perceptions of timeliness on the 
TP. There were several cases of users who stated they would use the TP more if it reliably 
published data on time. Users interested in historical data, including academics, found 
that data were published in an acceptable timeframe. However, these users mentioned 
that data updates were not marked as such and obscured historical information. 


Figure 20. Percentage of users assessing timeliness for different data domains. 
Key point: For every data domain, fewer than 40% of users reported that data were always there 
when needed. 
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Notes: Data domain names with asterisks represent those for which fewer than 30 users responded. 


Users interested in real-time data found that TSOs often published data on their websites 
more quickly than they appeared on the TP. 
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(a) Outage data and UMMs 


Information on planned and unplanned outages of generation and production units 
(15.1.A-D), consumption units (7.1.A&B) and the transmission grid (10.1.A-C) are 
reported in the form of Urgent Market Messages (UMMs). These messages must identify 
the concerned units or transmission assets, the start- and end date of the event and the 
installed and available capacity during the event. The UMMs must be published as soon as 
possible, i.e. no later than one hour after an unplanned outage occurred or after a 
decision is made regarding a planned outage. Each outage as well as updated information 
is published separately as a new UMM. Whereas the UMMs can be downloaded individually 
for generation and production units and transmission assets, unavailabilities of 
consumption units are aggregated by bidding zone or control area. 


Most market participants whom we interviewed identified UMMs as an area of concern for 
a number of reasons: many users are missing a versioning scheme or timestamp that 
would allow coherently connecting the individual UMMs. There is no functionality to allow 
generation plants to combine information on a single power plant into one “profile” or 
message. Furthermore, users were concerned about duplicate UMMs reported for single 
generation facilities, inconsistencies when trying to download UMM information via API 
and FTP, slow reporting times on the part of power plants, inconsistent information 
compared to other data sources, information that was not provided in a useful order and 
country-specific issues in Belgium, Germany, the UK, Italy and the Netherlands. 


Publication requirements for outages and other UMMs stem not only from the 
Transparency Regulation but also from Regulation (EU) No. 1227/2011 (REMIT). As a 
consequence, UMMs are reported not only on the TP, but also on so-called Inside 
Information Platforms?? (e.g. EEX Transparency or Nordpool REMIT UMM). Some market 
participants expressed dissatisfaction that the TP is not intended to be an inside 
information platform. 


(b) Overwriting forecasts 


Another issue mentioned by users is that data are overwritten by updates even though 
the historical values also may be relevant for analysis. Users complained of historical data 
being overwritten by updates without an indication of whether there was an update, when 
it was made and where historical data would be available. Although historical data values 
must be archived according to Article 3.1 of Regulation 543/2013, they are not accessible 
on the TP for users. One example, as identified in a presentation entitled "Manual of 
Procedures Revision" given at an ETUG meeting, is "Day-ahead Generation Forecasts for 
Wind and Solar" (14.1.D). As it exists now, day-ahead 18:00 forecast values are 
overwritten by intraday 8:00 and then possibly more recent forecasts. 


In its summary of user feedback to ETUG, ENTSO-E identified not only the overwriting as 
an issue but also that no indication was given on the TP of such revisions, a comment 
repeated by our interviewees. The most prevalent concern was that day-ahead 
"Scheduled Commercial Exchanges" (12.1.F) are not reliably available as they are 


28 http://data.europa.eu/eli/reg/2011/1227/0j 
? https://www.acer-remit.eu/portal/list-inside-platforms 
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overwritten with intraday scheduled flows instead of reporting day-ahead and intraday 
values separately as had been the case prior to the introduction of the TP in 2015. This 
reportedly will be rectified in the latest MoP update. 


(c) Delays in data availability 


Users have singled out data from Belgium, Bulgaria, Ireland, Italy and the Netherlands as 
noticeably delayed. 


3.1.2.4 User friendliness 


User friendliness is different from completeness, accuracy and timeliness in two ways: 
first, it concerns the platform itself, rather than the data contained in it. Second, it is 
more subjective and its assessment cannot be accomplished by a statistical analysis of 
the data. Nevertheless, through our own experience during this and previous research 
and consulting projects, through reviewing assessments and through the online survey 
and the interviews we conducted, we are confident that we are able to report a 
representative and robust picture of which aspects of the TP users find satisfying and 
where they see the need for improvements. We have categorized comments regarding 
user friendliness into six broad categories: 


e Website and GUI, including ease of finding data, data presentation and ease of 
accessing data for downloads; 

e Automatic downloads, including useful documentation and reliable access; 

e Data files; 

e Displaying data availability, including why data is unavailable; 

e Data documentation and 

e User support. 


Of these, data documentation received the largest number of critical comments. 


(a) Website and GUI 


Navigating the website 


According to the ETUG user survey, about 71% of TP users find navigating the website at 
least moderately easy. Our own Survey revealed similar results; as displayed in Figure 21, 
users rated finding data on the TP an average of about 3.1 (1-5 scale; 5 intuitive). We 
find the grouping of data items on the website to be not always intuitive for first-time 
users; one example is “Day-ahead Prices” (12.1.D), which is categorized under 
“Transmission”. 
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Figure 21. Number of users rating intuitiveness of finding data on TP. 
Key point: Few users report finding data on the TP completely intuitive but on average, users find it 
fairly intuitive. 
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Response time and download speed 


Several users, both in interviews and via the online survey, reported slow server response 
and frequent time out errors, an issue also reported by the ETUG user survey. This is not 
only an issue of convenience, but can jeopardize functionality: if the TP is unstable, it is 
not always possible to scrape data as soon as it is posted. 


Final report 


Figure 22. Number of users rating speed of server response times. 
Key point: Users report server response to be slow. 
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Data selection and filtering 


Users find it difficult to download the exact data they want. Several users would like more 
and better implemented options to display and filter subsets of the data, an issue also 
reported by the ETUG user survey. A related need is the possibility to download data from 
multiple geographic entities at once rather than being restricted to one at a time—the 
lack of this option is an obstacle for users who must rely on the GUI to download larger 
amounts of data. 


Other website issues 


When navigating the website, users often are presented with tables not showing any 
values. The reason is not necessarily that data are missing: the GUI allows selecting all 
sorts of combinations of data item, geographical entity, point in time and possibly other 
(data item-specific) criteria for many of which data are not expected. This seems to be 
the case for many of the default views shown upon first selecting a data item. Another 
issue stems from the fact that users must log in before being able to download data from 
the GUI: if they first navigate to the data they are interested in and then decide to log in, 
they are returned to the homepage and must find the previously accessed data item once 
again. Users also suggested making the website more visually appealing and graphically 
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oriented; one best-practice example cited in the survey was the Fraunhofer ISE website 
Energy Charts”. 


(b) Automatic download 


The web interface (GUI) is one out of six ways to download data. Expert users who have 
implemented access via FTP or the Restful API expressed satisfaction with these two 
download options—one interviewee said the Restful API “is what works best about the TP 
website.” [2] 


However, users with less experience or without an in-house IT department supporting 
them often had trouble implementing automatic access. Maybe more problematically, 
many users are not aware that these options exist, likely because automatic download 
options are mentioned only in one of several FAQ collections on different subdomains of 
the ENTSO-E website. In the case of the FTP, ENTSO-E does not publicize it as a 
download method because it is in a test phase. 


In our user survey, users rated the usefulness of the GUI on average about 3.3, the FTP 
on average about 4.1 and the Restful API on average about 3.9 (1-5 scale; 5 very 
useful). Figure 23 displays how users rated these three download options. FTP and API 
download options were reported as very useful by nearly half of users. However, fewer 
users were familiar with FTP and API options than GUI. 


30 


https://energy-charts.de/ 


Final report 


Figure 23. Percentage of users rating usefulness of download options. 
Key point: FTP and API download options were reported as very useful by nearly half of users. 
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option. 


The data websites Quand?! and Kaggle?? were mentioned in the survey as best-practice 
examples of platforms with a focus on data integration, automation and speedy 
processing. RTE and Nordpool were also mentioned as positive examples. Overall, users 
were happy with FTP and API download options, although there were issues including 
unreliable availability of all data items and an inconvenient process for downloading 
updated values via API, overly technical API documentation and missing information via 
FTP. 


(c) Issues with data files 


Users are split in their opinion on XML files; some find them useful while others find them 
inconvenient. For all file types, file-naming conventions could be improved, which today 
include "spaces and uppercase and lower-case letters all in the mix", as one survey 
respondent wrote. It was also suggested to allow CSV users to choose date format, field 
separator and time period. When downloading CSV or XLSX data, data in 15-min, 30-min 
and 60-min resolutions are provided in one single file, which can be burdensome to work 
with. [11] [14] 
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(d) Displaying data availability and “master data” 


As there is no central area/data matrix, users often spend time clicking through to see 
whether a data item is available in a given area. This might be the most limiting 
shortcoming for first-time users: there is no easy way to get an overview of what is 
available on the TP. 


In their February 2017 opinion on the first revision of the MoP, ACER requested that a list 
of data providers per data item and geographical area be provided. 


An independent but related issue is the fact that the so-called Reference and Master Data 
are not available to users. In their opinion on the MoP update, ACER mentions bidding 
zones, control areas and borders and a list of generation units as examples of such data. 
In ACER's opinion, making this information explicitly available to users would complement 
existing download options. This was also brought up by interviewees, who were interested 
in maps of bidding zones, control areas and borders (and how these differ from one 
another) as well as data sources. 


(e) Data documentation 


Users mentioned unclear, insufficient and hard-to-find data. One user summarized that 
“there are some stubs available on the website, but in general the documentation 
is..poor: what is ‘load’? Which power plants are counted in a specific fuel-type 
[16.1.B&C]?" This is in line with user feedback reported in the ETUG survey: "more 
detailed data information was consistently suggested (close to data items/centrally): 
improved data definitions, methodologies, publication times, possible disclaimers, why a 
data item is not expected, contact info for data providers, matrix of data by provider/data 
availability". 


Another issue is the difficulty of accessing data definitions. In our user survey, nearly 
60% of respondents had not heard of the Detailed Data Descriptions. Without 
investigation, users also do not learn about the fact that the existence, content and 
governance of the TP are due to Regulation 543/3013, as it is not mentioned on the 
website. Numerous TP users said that documentation and metadata are difficult to find. A 
consensus among experts and other users was that documentation and metadata should 
be available from the same place as the data items are. 


(f) User support and contacts 


The TP does not publish contact details for or the identity of Data Providers or Primary 
Data Owners. All requests are channelled through the TP service desk, which forwards 
questions to Data Providers or Primary Data Owners. The service desk then responds by 
email to the requestor. This procedure has several shortcomings: 


e As noted above, other users are not warned about quality issues. This causes 
users to be unaware of existing problems and could lead to multiple requests 
about the same issue. 

e If addressed through the service desk, TSOs are often hesitant to reply, because 
on the grounds of non-discrimination they are not allowed to share information 
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with a market participant exclusively. However, they rarely make such information 
available to the public via their own websites, either. [1] [survey] 

e While most requests are addressed within a few days, some remain unanswered 
for several weeks without explanation. 


3.1.3 Conclusions and recommendations 
3.1.3.1 Priorities 


Missing, inaccurate or inconsistent data affect all types of users. Beyond this, different 
users have different requirements and priorities. “Light users" have different 
requirements and face different problems than “frequent users”. This section provides a 
list of issues by user type. 


(a) Light users 


Light users access the TP once or a couple of times per year. They are researchers or 
analysts who do a one-time assessment for which they require TP data. They use the GUI 
to download data manually. According to ENTSO-E, there are 8800 users registered for 
the TP. Given this large number, it seems plausible that the majority of these are light 
users. The main problems they encounter seem to be the following: 


e Problems with the GUI, including unintuitive navigation, slow response times and 
error messages, lack of filter options and lacking possibility to download multiple 
countries at once; 

e Hard-to-find data descriptions and documentation; 

e Lack of a central area/data matrix that indicates data availability; 

e Lack of information about automatic download options and 

e Long historical time series are missing and pre-2015 values are not integrated. 


(b) Frequent users 


Frequent users access the TP on a regular basis, sometimes multiple times per day, 
usually through the Restful API. They are often large market participants who also are 
obliged to provide data. Their companies often have IT departments that support their 
gaining automatic access; often, TP data are retrieved automatically and integrated into 
an internal database. Some market participants have dedicated staff or even teams 
working only on transparency data. Such users also may be members of ETUG. Through 
ETUG meetings, they are informed about the structure, problems and processes of the 
TP. The problems that frequent users encounter include the following: 


e Inconsistent interpretation of data definitions by different Data Providers; 

e Confusing outage data and UMMs as well as cross-border transmission flows and 
schedules; 

e Data are often used for close-to-real-time decisions (trading, dispatch), so 
timeliness in general and 

e Users expect to be able to use the TP as their primary source of information for 
outage data/UMMs but they cannot because the data do not satisfy the REMIT 
requirements. 
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3.1.3.2 Suggestions for improvements 


The individual issues outlined in the sections on completeness, accuracy and timeliness 
should be improved upon. The objective should be to increase completeness of time 
series data such as load, prices and generation to 100%. Inconsistencies with other data 
sources should be resolved or explained. The provision of event-driven data, i.e. outage 
data (7.1.A&B, 10.1A&B, 15.1.A-D) should be improved due to its importance for market 
ac-tors: the messages should be displayed in a sensible order, duplicates are to be 
avoided and older versions of UMMs should be retained and not overwritten. All this could 
be achieved by implementing a versioning system or by including a timestamp indicating 
when the message was published. Furthermore, it should be possible to display all 
outages for a particular unit in one place as implemented e.g. at Nordpool REMIT UMM.?? 


In the following, we present further suggestions for improvements that do not pertain to 
individual data items, but rather to cross-cutting issues related to usability, incentives 
and governance. 


(a) Improve information and navigation 


The issue. An issue for many users, especially those who are not among the large 
utilities that participate in ETUG, is a lack of well-structured information on the TP. 
Information is available, but it is scattered throughout the website, cannot be found 
through search engines and is sometimes buried in PDF documents. Navigation on the 
website can be unintuitive and makes sense only once one knows the legal background of 
the TP. Some information is available only to ETUG. 


Our proposal. The landing page of the TP should include an introductory text explaining 
the purpose of the TP and the fact that its existence, content and governance are due to 
Regulation 543/3013. We furthermore propose making information available where users 
look for it; making all ETUG-only information available to the public, including ENTSO-E's 
continuous quality assessment and providing a well-maintained, easy-to-find and 
searchable Q&A page that includes all data definitions. We recommend working with a 
specialist in search engine optimization to make sure that web search engines can crawl 
and index the Q&A page and rank it highly in search results. Detailed Data Descriptions 
should not be more than one click away from the data they refer to and vice versa. In 
addition, we propose introducing a public help forum to replace the bilateral service desk 
procedure, as many online product providers have done (e.g. Google Product Forums?^); 
see also following subsection. 


(b) A crowd-sourced public data quality log 


The issue. Users who believe they have identified inaccurate or missing data are 
supposed to contact the TP service desk, which then checks and/or communicates with 
the Data Provider. Other users are not made aware of the reported issue. 


53 https://umm.nordpoolgroup.com 
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Our proposal. We propose establishing a public data error log. Registered users should 
be able to post an item on the list if they encounter issues with completeness, accuracy 
or timeliness of data or with the usability of the platform. The TP service desk, the Data 
Provider and other users can respond and comment; all comments are public. Once the 
issue is solved the service desk flags the item as “solved”. The posting and comments 
remain online. Such a crowd-sourced public data log has multiple benefits: 


e Users are warned about issues and can use data with additional care. 

e Data providers are warned immediately about issues and have the chance to 
respond quickly. They also can explain that there is not an issue if that is the case. 

e Other users can post solutions or explanations. 

e A log creates transparency about structural problems and hence provides an 
incentive for Data Providers to improve the quality of their data and processes. 

e Itis a great way to source users’ ideas for improvements. 


(c) Automatic quality reporting 


The issue. There is no public automatic reporting on completeness, accuracy and 
timeliness. 


Our proposal. We propose having ongoing, regular and public reporting on at least 
completeness and timeliness (and maybe some aspects of accuracy). It should be easy 
for users to learn which data items are complete and whether recent additions have 
arrived on time. The reports should be linked prominently on the TP landing page and be 
accessible from each data item directly. Ideally, this table also would list the reason for 
the problem. It is our understanding that automatic quality reporting is a capability the TP 
already has; however, it is used only internally, possibly due to push-back from Data 
Providers. Such reporting would be complementary to the user-generated data quality log 
suggested above and share the same benefits. 


The combination of a user-reported data quality log and automatic reports would not fix 
all problems relating to data quality, but it would save users time and could work as a 
form of accountability for those parties that fail to provide high-quality data. 


(d) Machine-readable metadata 


The issue. Metadata are data (information) that provide information about other data. 
Three types of metadata exist: descriptive metadata, structural metadata and 
administrative metadata. It is our understanding that metadata, including sources, 
release dates and licences, are not available in many cases (e.g. XLSX, CSV download via 
GUI or FTP) in a machine-readable form. 


Our proposal. We propose providing metadata in JSON format as a complement to XLSX 
and CSV files for every data item and applicable geographic entity. Metadata should 
include at least the following information: unit of measurement, data source (Primary 
Data Owner), Data Provider, contact person, licence, link to Detailed Data Descriptions 
and—if applicable—to further (data item and/or data source-specific) documentation. We 
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recommend considering whether the CSV files could be organized to comply with the 
Tabular Data Package standard?? published by Open Knowledge. 


(e) Governance, ownership and incentives 


The issue. To us, the governance structure of the TP seems to be the underlying cause 
of many of the issues discussed above. Dispersed ownership and lack of incentives seem 
to lead to little attention to users. To us, it seems that responsibility and accountability 
are lacking: 


e ENTSO-E points out that it maintains the technical database; all data quality issues 
are a matter for Data Providers. 

e Data Providers are hard to contact and, to our knowledge, face no material 
incentives to improve quality. 

e National Regulatory Authorities (NRAs) apparently lack the capacity or the 
incentive to monitor data quality properly and to impose sanctions on non- 
complying Data Providers. 

e ACER lacks the mandate and the capacity to monitor data quality continuously; in 
addition, ACER recommendations are not binding for ENTSO-E. 


Our proposal. Building a useful power system data platform is a complex task. It can 
never be a one-shot project, but rather requires intensive improvements over a long 
period. It is burdensome and costly. We recommend improving incentives through 
transparency and—ultimately—sanctions and adapting the governance structure to focus 
more on users. We recommend that: 


e ENTSO-E get a clear mandate to specify data definitions further to improve 
consistency among Data Providers. 

e Users be able to publicly report issues. 

e Data quality be systematically monitored, with reporting by Data Providers and all 
monitoring reports made public. 

e NRAs receive yearly reports about compliance of all Data Providers of their 
jurisdiction. These reports should be public as well. 

e At some point, Data Providers face monetary sanctions for non-compliance with 
quality requirements and submission deadlines. If NRAs are responsible for 
imposing such sanctions, the size of the penalties should be public. 

e ETUG's role be expanded and formalized. 

e Users beyond market participants—in particular, civil society and academia—be 
represented formally in ETUG, following the spirit of the Aarhus Convention?6, 


The objective of this study was to evaluate the quality of data published through the 
ENTSO-E Transparency Platform as well as the user friendliness of the platform itself. By 
providing this analysis and suggesting the above improvements, we hope that we can 
help the TP become even more useful for its users. 


35 http://frictionlessdata.io/ 


36 http://ec.europa.eu/environment/aarhus/ 
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3.2 Output 2: Terms on data management and access 


3.2.1 Objective and approach 


This section presents a summary of a legal opinion that assesses legal aspects of using 
energy data published on the ENTSO-E Transparency Platform (TP). The full legal opinion 
authored by lawyer Till Jaeger of the law firm JBB Rechtsanwalte, is available as an Annex 
to this report. 


This output addresses the following questions: 


Is the TP protected under copyright or the sui generis database right? 
What kind of use is currently legally permitted? 

Who is the rightholder? 

How can legally reliable reuse of TP data be granted in the future? 


3.2.2 Findings 


3.2.2.1 Legal protection of the Transparency Platform database 


Defining “database”. The Database Directive 96/9/EC contains the statutory framework 
for the protection of databases. Databases are defined as a “collection of independent 
works, data or other materials arranged in a systematic or methodical way and 
individually accessible by electronic or other means.” (Note that this refers to the 
collection of data, rather than the software or hardware used to store it.) It seems 
evident that the data provided on the TP are a database in this sense. 


Protection of databases. The protection of databases is twofold: 


Copyright (Article 3). Databases which, by reason of the selection or arrangement 
of their contents, constitute the author's own intellectual creation shall be 
protected as such by copyright. As the transparency regulations specify which data 
have to be selected for publication and how to arrange them, there is little scope 
left for Data Providers for unique intellectual creation resulting in classical 
copyright protection. Hence, copyright protection seems unlikely. 

Sui generis right (Article 7). Independently, the maker of a database that shows 
that there has been a qualitatively and/or quantitatively substantial investment in 
either the obtaining, verification or presentation of the contents is granted a so- 
called "sui generis database right" or simply "sui generis right". This right was 
newly introduced at that time and does not exist outside the EU. The sui generis 
database right is similar to copyright; however, it is not granted for an intellectual 
creation but for the financial and professional investment made in obtaining and 
collecting the contents. This seems to be fulfilled in the case of the TP. Thus, any 
person who wants to reuse it has to assume that the database is protected by the 
sui generis database right. This protects the maker of a database from "extraction 
and/or re-utilization of the whole or of a substantial part" of it. 


Conclusion. The energy data provided on the TP constitute a database and are protected 
under the sui generis database right. 


57 


3.2.2.2 What kind of use is currently allowed 


Use cases. Different types of users use the TP for different purposes, ranging from 
electricity trading to system planning, research and policy advice. They also might use 
the TP in different ways: studying raw data, running statistical analysis of data, using 
data as input in computer models and/or sharing raw or processed data with others. 


Assessing legal use. The maker of the database has the exclusive right to reproduce 
and distribute the database and to make this available to the public. Therefore, and as a 
default, any such use is not permitted except when (i) a statutory limitation on the sui 
generis database right applies or (ii) the maker of the sui generis database grants a 
license to the user. Therefore, any use case has to be evaluated in three steps: 


e Is the respective act of reusing the data covered by the exclusive rights of the 
rightholder (or is the use outside the scope of the protection)? 

e Are any statutory exceptions to the sui generis right applicable (e.g. some kinds of 
private use, scientific research)? 

e Is the respective act of reusing the data allowed under the license provided by the 
maker of the database? 


Covered by protection? When downloading data from the website of a Data Provider, 
the downloading person is creating a new copy of the data. This reproduction affects the 
rights of the owner of the sui generis right if the database is copied as a whole or a 
qualitatively or quantitatively substantial part of the database is copied. 


Statutory exceptions applicable? Some Member States, notably Germany, permit by 
means of law downloading substantial parts for private or scientific use for non- 
commercial purposes. Differentiating “private” from “commercial” use is notoriously 
difficult. Private use has been interpreted very narrowly by courts in the past. In any 
case, it is clear that data use by companies (be it utilities, financial actors or 
consultancies) as well as contract research - even if conducted by universities — does fall 
under the scope of “commercial purposes”. In particular, it seems evident that all market 
participants are using TP data for commercial purposes. Therefore, it seems certain that 
exceptions from database right protection apply to a minor share of TP users at best. 


Allowed under licence granted? For the vast majority of TP users, the only way to 
legally download the TP database is if they are granted a licence. In this context, “licence” 
means the grant of a right to use the database with the scope of the right specified in the 
licence. The “General Terms and Conditions for the Use of the ENTSO-E Transparency 
Platform” is governing the use of data published on ENTSO-E's website. This is not a 
classical licence agreement but it stipulates which uses shall be allowed. There is no 
explicit clause for a grant of rights but the permission. However, one could argue that it is 
implicitly assumed (“when using of the Transparency Platform Data for any purpose 
whatsoever”). Therefore, it remains unclear to which extent a re-use is permitted. As a 
consequence, most users cannot be sure that they are currently using the TP database 
legally. 


Other use cases. Usually, downloading data is only the first step of using it. Other types 
of use, in particular making it available to others (through the internet or a company’s 
intranet), do fall under the scope of the database right, as well. While the legality of the 
download itself remains unclear, it is very clear that ENTSO-E does not provide users with 
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the right of any other use. In particular, using the TP database as part of an Open Data 
project that requires publishing data under an open license is not permitted under the 
license granted by ENTSO-E. 


Conclusion. Most use of the TP is likely to be commercial—including all use by market 
participants but also contract research conducted by universities. Statutory exceptions do 
not apply to commercial use. Therefore, using a “substantial” part of the TP database only 
would be permitted if a licence were granted by the maker of the database. This is 
currently not the case, as ENTSO-E does not grant such use rights without doubt and the 
scope of allowed usages remains unclear. As a consequence, most use of the TP going 
beyond the mere download can be seen as constituting a copyright infringement. In other 
words, most users cannot use TP data in a legally safe manner. 


3.2.2.3 Who is the rightholder? 


Multiple rightholders. ENTSO-E points out that the “Primary Data Owner” of the data 
may be the rightholder of the sui generis right. According to our understanding, the data 
published on the ENTSO-E Transparency Platform stem from several sources and are 
stored in several differing databases: 


e Transmission system operators, power plant operators and other Primary Data 
Owners 

e Energy exchanges and other “intermediaries” (which might be at the same time 
Primary Data Owners) that serve as Data Providers 

e ENTSO-E 


According to our analysis, it seems likely that all three actors are making a substantial 
investment in either the obtaining, verification or presentation of the contents of the TP 
database. As a consequence, they all hold sui generis right on (parts of) the database. 


Implications. Consequently, third parties interested in the reuse of the data would need 
the permission of the Primary Data Owners, the permission of the Intermediaries and the 
permission of ENTSO-E. Taking into account the significant number of Primary Data 
Owners and the missing consent of the rightholders about under which licence conditions 
such data shall be publicly available a legal reuse of the database of ENTSO-E is 
practically impossible. Additionally, if one or more rightholders refuse to license their 
data, an interested party would have to exempt such data to be allowed to reuse the 
remaining parts for which the respective rightholders allowed the use. To provide a 
practical example: if ENTSO-E would like to license its database under a Creative 
Commons licence for free reuse, the permission of all rightholders would be needed. If 
one or several Primary Data Owners refuse such licensing, ENTSO-E would have to 
identify the respective data and restrict the Creative Commons licence to the remaining 
part of the database. The parts of the database licensed under the Creative Commons 
licence need to be identified as taking part in such licensing to allow practically the reuse. 


Conclusion. It seems likely that both ENTSO-E and Primary Data Owners and possibly 
intermediaries that have transferred the data hold sui generis rights on the TP database. 
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3.2.3 Conclusions and recommendations 


Provide no licence. If ENTSO-E would drop all references to copyright and related rights 
from the Terms of Use of the TP, all rightholders would still retain their rights. In 
particular, downloading or re-using data otherwise beyond the existing statutory 
exemptions would not be legally possible. 


Directive on Copyright in the Digital Single Market. In 2015, the Commission 
released a Communication entitled “A Digital Single Market Strategy for Europe”, followed 
in 2016 by a proposal for a “Directive on copyright in the Digital Single Market”. As 
proposed, however, the Directive would not introduce new statutory exemptions that 
allowed legal use of energy data. 


Amending the Copyright Directive. New statutory exemptions could be included in 
Directive 96/9/EC. Such an amendment would be complicated and seems unrealistic. 


Open data licensing. We believe the best option for legally reliable reuse of TP data is 
open data. “Open data” is the idea that data should be available to everyone to use and 
republish without restrictions of usage rights and license fees. Whether a database can be 
considered open data depends on the licence applicable. A number of such licences exist, 
one example being the Creative Commons Attribution 4.0 (CC-BY-4.0) licence. For this to 
happen, all rightholders have to grant an open (inbound) licence to ENTSO-E, which then 
can grant an open (outbound) licence to users. 


Conclusion. The best way forward seems to be to grant an open licence of the TP 
database. This requires, however, consent of all rightholders. 
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4. FINDINGS FOR WORK PACKAGE 2 


Work Package 2 was implemented by Copenhagen Economics and VVA. VVA was 
principally leading the data collection for Outputs 3, 4 and 5, while Copenhagen 
Economics was leading the analysis of the results in Outputs 3, 4, 5, and completed 
Output 6. 


Security of electricity supply is a key concern for European Member States, and a concern 
that has spurred a number of countries to pursue subsidies to put forward capacity (as 
opposed to energy) to the market. Security of supply is however not just about having 
enough generation capacity available (adequacy in the literature), but also about having a 
well-functioning grid that is robust to different types of stress such as congestion and 
physical failure through e.g. damaged cables or lines. In stress situations that are at risk 
of evolving into a disruption, certain policies or measures such as voluntary demand 
curtailment can help reduce the stress in the grid, and therewith avoid the disruption. 
Every time there is lack of electricity supply, there is a loss to society through avoided 
consumption possibilities. 


While important, there has so far not a clear picture of the extent of disruptions nor their 
causes, potential mitigation and their market impact. This study will provide a better 
understanding of those issues, which are important in policy formation: 


Regarding their causes: 

e Are disruptions primarily caused by lack of generation? 

e Are disruptions primarily caused by grid issues? 

Regarding their mitigation: 

e Are voluntary demand curtailment programmes effective solutions to avoid 
disruptions? If so, their initiation could be promoted and/or accelerated across Europe. 
If their effectiveness depends on the design of the programmes, then the effective 
design should be promoted. 

Regarding their socioeconomic impact: 

e How much harm do disruptions do to consumers in Europe? The value of lost load 
reflects the welfare loss caused by a disruption and is therefore an indicator for the 
scale of the problem. 

To answer those questions, we will in this work package analyse how outages, 

disruptions, voluntary demand curtailment and the value of lost load are linked to each 

other. In this study, we use the following definitions: 

e Outages (output 3) are events where a power generation or production unit is out of 
service. Outages can be planned, for example for the maintenance of units, or forced, 


for example as a consequence of technical malfunctions in the unit. Outages can, but 
do not necessarily, lead to a situation where consumers are out of power. 


61 


We also consider malfunctions of interconnectors as outages. The reason is twofold: 
firstly, interconnectors increase the capacity available to a market and are in that role 
more similar to production units than to the transmission or distribution grid. Secondly, 
malfunction of an interconnector does not necessarily entail a situation where 
consumers are out of power, and can therefore not be called a disruption following the 
definition used in this study. Here, again, interconnectors show similarity to production 
units. We will however treat interconnector outages separately as they cannot be 
clearly allocated to one country. 


e Disruptions (output 4) are situations where consumers cannot consume the amount 
of electricity they would like to demand; such situations typically mean that consumers 
are out of power, as in blackouts or brownouts. Disruptions can happen at electric 
underground cables, overhead lines or transformer stations, each either on DSO or 
TSO level. As described above, we do not include malfunctions of interconnectors as 
disruptions, but as outages instead. 


e Voluntary demand curtailment (output 5), often referred to interruptible contracts, 
are agreements between electricity consumers and electricity supplier and/or TSO/DSO 
that allow the latter to curtail the electricity delivered to this consumer (or to cut it 
completely) in case of emergency. 


e The Value of Lost Load (output 6) is an indicator for the welfare loss due to a 
disruption. It is provided in EUR per kWh of non-consumed electricity due to the 
disruption. 


The following figure reflects the overall relationship between outages (output 3), 
disruptions (output 4), voluntary demand curtailment (output 5) and value of lost load 
(output 6). In the following chapters, we will disentangle and test those relationships. 


Figure 24 : How disruptions are related to outages, voluntary demand curtailment and 
the value of lost load 
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4.1 Output 3 - The extent and source of electricity outages 


4.1.1 Objective and approach 


The objective of Output 3 is to provide information electricity outages in European 
Member States and their characteristics. 


The data on outage includes information on plants (generation units), the nature of 
outages (planned or unplanned and the cause), their length in time, the reduction of the 
available capacity and the amount of non-generated electricity (by Member State and 
generation technology on at least a monthly basis). The information was collected in 
three steps. First, we downloaded and analysed the outage data from the ENTSO-E 
Transparency platform (TP), which constitutes the bulk of the data obtained. The limit for 
reporting outage data to the transparency platform is 100 MW,?/ so we have limited 
information about outages below 100 MW. Then we carried out desk research on 
publications from ENTSO-E and from electricity providers' websites. To complement the 
information, we launched a survey with ENTSO-E TP data providers, TSOs and electricity 
providers, and received responses from 18 Member States. 


4.1.2 Findings 


4.1.2.1 Overview and overall extent of the issue of outages in Europe 


The reported outages per year and the capacity reduction and non-generated electricity 
associated with it vary considerably across European countries, as the following figures 
Figure 25, Figure 26 and Figure 27 show. Important to note, those figures show absolute 
values that are not scaled to the countries size or installed capacity. A detailed overview 
on all figures, absolute and scaled, can be found in Table 2. 


In the period of 2015 to 2016, the United Kingdom reported the highest absolute number 
of outages with almost 6,500 outages per year, Germany coming second with 3,750 
reported outages see Figure 25. The other Member States report much fewer outages, 
down to less than ten per year like Bulgaria and Luxembourg??. The EU28 median number 
of outages per year is 249. Considering the installed capacity in the country, which 
reflects the country's size, Malta has the highest number of outages, see Table 2. 


?' Following the Commissions Regulation No. 543/2013. This threshold is to a large extent to reduce 


compliance costs, to prevent administrative costs of reporting very small outages. With increasing 
deployment of renewable energy sources that are often below 100 MW, and potential closure of many of the 
traditional power plants, reported outages above 100 MW is likely to capture a smaller amount of the total 
generation fleet. Lowering the threshold would allow the outage statistic to capture more of the outages 
going forward, but would also constitute higher reporting and compliance costs. In addition, it can be noted 
that comparing outages over time will be less accurate if/as the generation fleet changes to a higher share of 
outages below the threshold, which are therefore outside the reporting obligation. 

Although reporting outages to the ENTSO-E Transparency Platform is obligatory following Regulation 
543/2013 since January 5, 2015, data gaps have been identified (see Work Package 1), and could only 
partly be filled through the survey to electricity producers and TSOs (see Annex). The possibility that low 
figures - like for Bulgaria - are due to incomplete reporting can therefore not be excluded with certainty. 
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Figure 25 : Average number of outages per year, 2015-2016 
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Note: The bar for the UK and Germany is not shown in its full size, as indicated by the white stripes, to avoid that the rest of the 
figure becomes too small and difficult to read. The number of outages for those two countries can be seen from the 
data labels. Croatia, Cyprus and Slovenia did not report any outages in 2015 or 2016. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform. 


Looking at the total reduction of available capacity due to outages per year, the United 
Kingdom lies again in the very top, with a total reduction of available capacity more than 
three times as high as the second country, Germany, see Figure 26. The median across 
Europe is a total reduction in available capacity per year of 60 GW. 


Figure 26 : Total reduction of available capacity per year, 2015-2016 average 
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Note: The bar for the UK and Germany is not shown in its full size, as indicated by the white stripes, to avoid that the rest of the 
figure becomes too small and difficult to read. The reduction of available capacity for those two countries can be seen 
from the data labels. Croatia, Cyprus and Slovenia did not report any outages in 2015 or 2016, and Malta did not 
report the outages’ reduction in capacity. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform. 


Combining the reduction in available capacity and the duration of an outage allows for 
calculating the amount of electricity that has not been generated due to an outage. This 
value varies from 6,600 GWh (United Kingdom) to 250 MWh (Luxembourg) in total, see 
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Figure 27. Seen in relation to the countries’ sizes, Hungary and Estonia have the highest 
index for non-generated electricity per year, see Table 2. 


Figure 27 : Total non-generated electricity per year, 2015-2016 average 
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Note: The bar for the UK is not shown in its full size, as indicated by the white stripes, to avoid that the rest of the figure 
becomes too small and difficult to read. The non-generated electricity for the UK can be seen from its data label. 
Croatia, Cyprus and Slovenia did not report any outages in 2015 or 2016. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform. 
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Table 2: Detailed figures on outages in Europe, 2015-16 


reduction of Jer men- 
number of reduction of : Total non- generated 
number of à available sat 
Gaunt te ee Der outages per available capacity [MW] generated electricity 
Y 9 » installed MW of capacity per os ans electricity per [MWh] per 
year [ capacity year [MW] [P | year [MWh] installed MW of 
MW of capacity t 
capacity 
Austria 71 3 22,099 1 198,763 9 
Belgium 249 12 48,903 2 495,304 24 
Bulgaria 6 0 1,494 0 55,932 4 
Croatia n/a n/a n/a n/a n/a n/a 
Cyprus n/a n/a n/a n/a n/a n/a 
Czech 1,177 58 134,901 7 423,239 21 
Republic 
Denmark 292 20 70,313 5 271,263 18 
Estonia 146 54 84,855 31 703,894 261 
Finland 133 8 27,582 2 173,887 10 
France 1,068 10 366,567 4 2,510,439 24 
Germany 3,750 18 966,673 5 1,904,903 9 
Greece 13 1 4,087 0 152,137 9 
Hungary 679 82 184,835 22 2,305,232 279 
Ireland 803 82 41,229 4 124,237 13 
Italy 555 6 213,746 2 3,310,171 35 
Latvia 58 21 10,388 4 61,308 22 
Lithuania 15 4 9,390 3 130,369 36 
Luxembourg 6 21 886 3 252 1 
Malta 96 158 n/a n/a 19,326 32 
Netherlands 446 14 155,180 5 274,775 9 
Poland 744 20 158,726 4 734,599 20 
Portugal 222 11 31,705 2 165,028 9 
Romania 186 8 35,425 2 528,408 23 
Slovakia 23 3 3,388 0 22,362 3 
Slovenia n/a n/a n/a n/a n/a n/a 
Spain 770 7 203,667 2 1,838,258 17 
Sweden 436 11 99,055 3 821,589 21 
United 6,464 110 3,404,937 58 6,602,081 112 
Kingdom 
En 249 12 59,608 3 274,775 20 
median 
EU28 total 18,402 742 6,280,026 170 23,827,757 1,022 


Note: All figures are per year (2015-16 average). Total installed generation capacity was not available for Cyprus and Malta, and 


was extrapolated based on their number of households. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform. 
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In total, outages in Europe caused 32 TWh of non-generated electricity in 2015, and 12 
TWh in 2016, see Figure 28.?? The figures for 2013 and 2014 are both significantly 
higher, although the data for those years is not complete. This clearly supports the 
hypothesis that there is a decreasing trend over time, or at least from 2013-2014 to 
2016. 


Figure 28 : Total non-generated electricity per year in Europe 
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Note: Croatia, Cyprus and Slovenia did not report outages in any of the years. 2015 is the first full year where the reporting of 
outages to ENTSO-E is mandatory; the data for 2013 and 2014 is therefore incomplete in the sense that it only shows 
the outages that were reported voluntarily. The real total non-generated electricity in those years might be much 
higher, meaning that the decline over time to 2016 is much more significant than it seems in this figure. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform. 


4.1.2.2 | Comparing planned and forced outages 


Outages can be planned or forced. Planned outages mean that the production unit is 
taken off grid in a controlled manner and on purpose, for example due to maintenance. 
Forced outages are non-controlled outages that can occur for example due to technical 
disfunctions. 


Whenever a planned outage is scheduled, arrangements can be made to make sure that 
the drop in electricity production does not cause problems in the transmission or 
distribution grid, for example due to too low supply; other production units can for 
example be ramped up in time. Those arrangements cannot be made for sudden, forced 
outages, which is why those are much more problematic and typically cause more stress 
to the grid. 


?? The data from the ENTSO-E Transparency Platform provides country-level information on outages. However, 


the reported outages in relation to the installed capacity in some countries seem surprisingly low, especially 
for Greece and Bulgaria, also in the years 2015 and 2016, where reporting has been mandatory. We followed 
up on this issue with ENTSO-E directly, and it seems that ENTSO-E cannot ensure with certainty whether the 
data is complete for all countries and years. Based on this, we decided not to draw conclusions from country- 
level comparisons. Conclusions can still be drawn from EU totals (which will be a conservative estimate) or 
shares per generation type, as there is no reason to assume that there is a bias between years or generation 
types. 
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Looking at all reported outages in the EU in the period of 2010 to 2016, the number of 
forced outages dominates: 55% of all outages were forced. On country level, Spain, the 
UK and Ireland have the largest share of forced compared to planned outages, see Figure 
29. In those three countries, more than 3 out of 4 outages are forced. 


Figure 29 : Share of planned vs forced outages in Europe, 2010-2016 
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Note: Croatia and Cyprus did not report any outages. Malta did not report whether its outages were planned or forced. Bulgaria, 
Greece and Italy did only report planned outages. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform and survey answers. 


Forced outages differ from planned ones not only in their impact, but also in their 
characteristics. An average planned outage in Europe in the period 2010 to 2016 lasted 
34 hours, while the duration of an average forced one was only a quarter of that, namely 
9 hours, see Figure 30. This large difference is reasonable, as there is no need to keep 
planned outages as short as possible. Planned outages are announced in advance, and 
replacements for the reduction in capacity can be found. There are also cases where 
production units go off grid for a longer period, for example combined heat and power 
plants during the summer, when the heat is not needed. Those very long planned outages 
will pull the average duration upwards. For forced outages in turn, there is an incentive to 
keep them as short as possible. 


The average reduction in available capacity is much higher for forced (410 MW) than for 
planned outages (260 MW) in Europe. A potential explanation here might be that with 
maintenance, parts of the generation unit (for example single blocks) can go offline while 
parts of the unit continue producing electricity. When forced, outages might hit the full 
unit. 


The reduction of available capacity combined with the duration of an outage result in the 
non-generated energy, which is 6,200 MWh for an average planned outage, and 1,800 
MWh for an average forced outage. 
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Figure 30 : Characteristics of an average outage in Europe, planned vs forced, 2010-2016 
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Note: Croatia and Cyprus did not report any outages. Malta did not report whether its outages were planned or forced. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform and survey answers. 


An average duration of 34 hours for planned outages can seem rather low against the 
backdrop that some maintenance work takes several weeks or even months. The reason 
for this average is that a large number of very short planned outages has been reported - 
e.g. more than 4,000 outages with durations below 5 minutes - see Figure 31. 


Figure 31 : Histogram over the durations of planned outages, 2010-2016 
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Note: Lower boundaries are included, upper boundaries excluded (i.e. an outage of exactly 4 hours will be shown in the 


category 4-8 hours, not in the category 2-4 hours). Croatia and Cyprus did not report any outages. Malta did not report whether 
its outages were planned or forced. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform and survey answers. 
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4.1.2.3 Comparing outages of different generation technologies 


Looking at absolute numbers, then most outages occur in production units using fossil 
fuels for electricity generation, in particular coal and gas. However, those generation 
types are also the ones with the most installed capacity across Europe. Accounting for the 
installed capacity, offshore wind causes most total outages, namely on average 331 per 
year per installed GW, see Figure 32. Most of those outages are planned ones. Fossil coal, 
peat and gas come second, third and fourth, with 129, 77 and 39 outages per installed 
GW on average. Those generation technologies have in common that their outages are 
dominated by forced down times. 


Figure 32 : Generation type shares of planned and forced outages, 2010-2016 
number of outages per installed GW 
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Note: Croatia and Cyprus did not report any outages. Malta did not report whether its outages were planned or forced. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform and survey answers. 


Production units using fossil fuels are also the main contributors to the total non- 
produced electricity in Europe. In 2015, outages at units using fossil coal, gas, oil, and 
peat led to 19 TWh of non-produced electricity, which is more than half of the total of 35 
TWh, see Figure 33. Fossil coal and gas units each account for about a quarter of the 
total. Hydro power units and nuclear power plants add another quarter. While the total 
non-produced electricity due to outages decreases significantly from 35 TWh in 2015 to 
13 TWh in 2016, the relative pattern remains almost unchanged: fossil fuel units make up 
more than half of the total, followed by nuclear and hydro power plants. Renewable 
energies (biomass, wind on- and offshore) cause less than 1% of the non-generated 
electricity in both years. 


Worth noting is that the non-generated electricity due to outages of fossil coal plants 
decreased significantly from 2014 to 2015. 
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Figure 33 : Non-generated electricity by generation type per year in Europe 
TWh (=1 mio MWh) 


50 z 47 ks incomplete reporting until 
41 1 (&including) 2014 
I 
40 35 


20 


10 


N 2013 2014 zt 2015 2016 
= Biomass m Fossil coal m Fossil gas m Fossil oil 
a Fossil peat Hydro m Nuclear m Wind offshore 
m Wind onshore mother n/a 


Note: Croatia, Cyprus and Slovenia did not report outages in any of the years. 2015 is the first full year where the reporting of 
outages to ENTSO-E is mandatory; the data for 2013 and 2014 is therefore incomplete in the sense that it only shows the 
outages that were reported voluntarily. The real total non-generated electricity in those years might be much higher, meaning 
that the decline over time to 2016 is much more significant than it seems in this figure. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform and survey answers. 


The non-generated electricity from forced outages makes up only a smaller share of those 
numbers. In 2015, less than 8 TWh have not been generated due to forced outages in 
Europe, and less than 3 TWh in 2016, see Figure 34. While outages of many different 
generation technologies in many different Member States have contributed to that sum, 
the United Kingdom is worth mentioning with extensive forced outages of fossil gas units 
(1,8 TWh of non-generated electricity) and fossil coal units (1,3 TWh of non-generated 
electricity) in 2015. 
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Figure 34 : Non-generated electricity (forced outages) by generation type and Member 
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Note: Croatia, Cyprus and Slovenia did not report outages in any of the years. 2015 is the first full year where the reporting of 
outages to ENTSO-E is mandatory; the data for 2014 is therefore incomplete in the sense that it only shows the outages that 
were reported voluntarily. The real total non-generated electricity in those years might be much higher, meaning that the 
decline over time to 2016 is much more significant than it seems in this figure. 

“other” in this figure contains all those categories with less than 5,000 MWh of non-generated electricity. In alphabetical order, 
that is: Biomass (Hungary, Belgium, Finland), fossil coal (Slovakia, Portugal, Sweden, Austria), fossil gas ( Sweden, Latvia, 
Lithuania, Poland, Finland, Czech Republic), fossil oil (Finland, Denmark), fossil peat (Finland, Ireland), hydro (Austria, Belgium, 
Luxembourg, Ireland, Latvia, Lithuania, Czech Republic), other (Netherlands, Ireland), Wind offshore (Belgium), Wind onshore 
(Sweden). 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform and survey answers. 
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There is a tendency that renewable energy sources cause relatively fewer forced outages 
than non-renewable energy sources, see Figure 35. Both wind (offshore and onshore’) as 
well as hydropower have less forced outages than planned ones; offshore wind features a 
remarkably low rate of forced outages of only 16 percent, which is however mainly driven 
by a very high number of planned outages, as we could see in the figure before. For fossil 
fuels, most outages are forced ones. 62 to 72% of all outages in generation units fired by 
fossil coal, gas and peat are forced. 


Figure 35 : Share of forced vs planned outages per generation type, 2010-2016 
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Note: Croatia and Cyprus did not report any outages. Malta did not report whether its outages were planned or forced. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform and survey answers. 


For how long an outage will last depends largely on two interrelated things: the 
generation technology as well as whether it is a planned or forced outage. Whenever an 
outage is planned, arrangements can be made in advance to compensate for the non- 
generated electricity, making sure that supply will meet demand. In cases of planned 
outages, it is therefore not necessarily prioritised to keep it as short as possible. There 
are even situations where fossil-fuel based generation plants are taken down for longer 
periods on purpose, for example combined heat and power plants in warm summer 
periods where the heat is not needed and where electricity consumption tends to be low. 


Figure 36 (upper figure) shows that fossil-fuel based generation plants have indeed the 
longest down times, with 42 hours on average per outage for coal, 24 for oil, 18 for peat 
and 17 for gas. The planned outages for wind parks in turn take only 1.3 hours and 0.4 
hours on average for onshore and offshore wind respectively. 


The picture for forced outages looks different. In cases of forced outages, no 
arrangements can be made in advance to compensate for the loss of generation, which 


^' Note that there are very few reported outages for onshore wind that is above 100 MW, so data availability is 
low. 
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means that one typically tries to fix those outages quickly and keep the duration short. 
The data shows that, as expected, forced outages tend to be generally shorter in duration 
than planned ones. The range for forced outages goes from 0.8 to 8.4 hours, as it was 
0.4 to 41.8 for planned ones. The data furthermore shows no longer higher values for 
fossil fuels when looking at forced outages; here, biomass features the highest with a 
duration of 8.4 hours on average per disruption in the period 2010-2016. 


Figure 36 : Average durations of forced (upper figure) and planned (lower figure) 
outages per generation type, 2010-2016 
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Note: Croatia and Cyprus did not report any outages. Malta did not report whether its outages were planned or forced. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform and survey answers. 


Combining the duration with the reduction in available capacity results in the non- 
generated electricity, which is typically provided in MWh. This number reflects how much 
electricity supply is lacking, and gives therefore particularly for forced outages an 
indication for the impact on the market. For planned outages, fossil coal and oil have the 
largest non-generated electricity per outage. This is to some extent due to their long 
average duration, and is not necessarily a problem, as other generators can fill in this gap 
if the outage is known well in advance. The non-generated electricity per forced outage is 
a more crucial indicator. Here, fossil gas and oil are highest with more than 1,800 and 
1,600 MWh of non-generated electricity per outage, see Figure 37. Nuclear generation 
plants come third right after fossil oil. From the figure before we know that the duration 
alone cannot entail those results, which indicates that if those technologies have an 
outage, then the affected capacity is quite large. 
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Figure 37 : Average non-generated electricity for planned (upper figure) and forced 
(lower figure) outages per generation type, 2010-2016 
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Note: Croatia and Cyprus did not report any outages. Malta did not report whether its outages were planned or forced. 


Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform and survey answers. 


Among the observations where the generation type has been provided (the data depicted 
in the figures above), planned outages clearly cause more non-produced electricity than 
forced ones. However, considering all outages - also those where the generation type 
was not provided, which are mostly forced - this conclusion turns around. In fact, of non- 
generated electricity due to registered outages 2010-16, 41% has been due to planned 
outages, and 59% due to forced ones. 


4.1.2.4 | Outages at interconnectors 


Seen from a country’s perspective, an interconnector can be thought of as similar to a 
generation plant: it increases the overall capacity available on the supply side of the 
power market. Similarly, an outage of an interconnector is also very similar to having an 
outage in a large generation unit. That is, capacity that was thought to be available to the 
market is no longer available. 
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In the period of 2010 to 2016, unavailabilities have been reported at 68 different 
international tie lines in Europe,** affecting almost all Member States.^? The total duration 
of interconnector outages in Europe amounts to roughly 70,000 to 105,000 hours every 
year, see Figure 38. Most outages in terms of duration have been reported in 2013, 
where international tie lines have been unavailable for 105,000 hours in total. That 
corresponds (mathematically) to about 630 weeks, or 12 years, meaning that at any time 
of the year, there were 12 interconnectors unavailable on average. There appears to have 
been an upward trend before, and a downward trend after 2013. 


Figure 38: Total duration of interconnector outages, 2010-2015 
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Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform. 


Similarly to generation units, an outage in an interconnector does not necessarily 
constitute a problem. For example, it might not have been the case that power would 
have flowed on the cable if it had been working. Nonetheless, when an interconnector is 
not working, it effectively removes a part of a country’s supply curve/merit curve, thereby 
making it more vulnerable for further outages in generation units. 


As data on the non-available capacity per outage and therewith for the non-transferred 
energy is not available, we can only use information on the duration of interconnector 
outages - which, too, is an important indicator and allows to draw founded conclusions. 
The data clearly shows that planned interventions such as maintenance, repair and new 
constructions (blue colours) account by far for a larger share of those hours than forced 
outages from overload, false operation, failure, outside impacts or very exceptional 
conditions (yellow/orange colours), see Figure 38. In fact, the latter seem to make up 
only a very small part, while maintenance alone accounts for more than 40% on average 
in the period 2010-2015. Unknown or other reasons (grey) have been reported 
frequently, too, which limits the conclusions that can be drawn regarding the causes. 


^! This data includes all international tie lines in Europe that report to ENTSO-E, also tie lines from or to 


European countries that are not EU28 Member States. 


42 With the exemption of Cyprus and Malta. The lack of data here could be due to incomplete reporting. 
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Considering the duration per reported outage, outages due to new constructions take by 
far the longest with almost 10 days on average. Repair and maintenance typically cause 
unavailablities of little over 3 days. All forced outages in turn tend to be shorter; a false 

operation is on average fixed in 3 hours, failures or outages due to outside impacts after 
less than 2 days. Outages due to very exceptional conditions or overload are the longest 
in this group; they take up to 3 days on average, see Figure 39. 


Figure 39 : Average duration per interconnector outage, 2010-2016 
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Source: Copenhagen Economics based on data from the ENTSO-E Transparency Platform. 


4.1.3 Conclusions 


In 2016, approximately 886 GW of power generating capacity were installed in the EU. 
Outages mean that not this full total will be available for production all the time. In fact, 
more than 14,500 outages have been reported in Europe each year (2015-2016 average), 
which corresponds to 17 outages each year per installed GW of production. 


When taking into account the length of the individual outages, we estimate that the total 
non-generated electricity due to outages in the EU in 2016 was about 13 TWh. This 
means electricity that could have been generated if the power plants had not been out for 
whatever the reason. This corresponds to the 886 GW of power generating capacity in 
Europe being completely out for around 15 hours in the whole year of 2016, or to 0.17% 
of Europe’s capacity being out constantly. This number is significantly lower than just a 
few years back, having been reduced from about 41 TWh in 2013, 47 TWh in 2014 and 
35 TWh in 2015. The decline is likely to be even higher than this, as data availability for 
2013 and 2014 was significantly lower than for 2015 and 2016. 


Forced outages constitute the majority of the non-generated electricity, about 59%, and 
they occur significantly more frequently. About 60% of all outages are forced events, 
however the duration of the outage is significantly shorter (about 9 hours on average 
compared to 34 hours for planned downtime). Moreover, forced outages typically happen 
for much larger assets than planned. Indeed, the average affected capacity for forced 
outages is about 410 MW compared to about 260 MW for planned outages. This suggests 
that larger generation assets are more susceptible to outages than smaller ones. 
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In terms of fuel-based generation assets, fossil-fuel assets have by far the longest 
duration for planned outages of about 20-40 hours on average, compared to less than 1 
hour for wind turbines, 5 hours for nuclear plants and about 7 hours for hydro plants. On 
the other hand, the fossil-fuel based generation assets seem to be around the average to 
bounce back from a forced outage, taking on average about 2.8 hours, similar to hydro 
and nuclear. That is faster than biomass but slower than off- and onshore wind assets 
using 8.4, 1.8 and 1.3 hours respectively. 


Wind turbines mark the two extremes regarding how prone they are to outages. Onshore 
wind has the lowest total number of outages with 2 per installed GW, offshore the highest 
with 331 outages per installed GW - of which most are planned ones though. Both on- 
and offshore wind are among the fastest generation technologies to be up and running 
again after an outage, both planned and forced. The combination of very few and very 
short outages for onshore wind may to some extent compensate for the fact that the 
utilisation of wind capacity is significantly lower than traditional plants due to varying 
wind load factors. 


Just as outages at generation units, unavailabilities of interconnectors limit the capacity 
available to a country. Interconnectors have been unavailable for roughly 87,000 hours 
every year (2010-2015 average), which means that 10 interconnectors are unavailable in 
Europe at all times on average. Works at the cables and lines, especially maintenance 
(otherwise repair and new constructions) account for the majority of those unavailabilites, 
while forced outages (due to overload, false operation, failure, outside impacts or very 
exceptional conditions) only play a minor role. Outages due to works at the cables and 
lines tend to take longer; the longest durations are due to new constructions, where an 
outage takes almost 10 days on average. All forced outages can be fixed much quicker, 
after 3 hours to 3 days on average. 


4.2 Output 4 - Electricity supply disruption events 


In this chapter we will investigate on the issue of electricity supply disruption events in 
Europe, their scale (how many, how long, how severe) and scope (how spread across 
Europe). We will analyse the reasons for and characteristics of those disruptions. 


4.2.1 Objective and approach 


The aim of Output 4 is to provide information on the extent of electricity supply 
disruptions in recent years in the EU, and the reasons for these disruptions. The overall 
objective is to enable the Commission to understand the context and circumstances of 
disruption events in the EU, including (but not exhaustive) in which countries they are 
most prevalent, and whether they are caused by grid-related incidents, lack of generation 
capacity or something different. 


The information was collected in two steps. First, we carried out desk research of 
literature and databases at EU level (e.g. ACER, CEER, ENTSO-E TP and Monthly Statistics 
Reports) and national level (e.g. publications of NRAs and TSOs). Then, we 
complemented the information with a survey with NRAs and TSOs and received responses 
from 25 Member States. 
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4.2.2 Findings 


Electricity supply disruptions can occur from two sources: the lack of generation to meet 
demand, or problems with the infrastructure (reliability). The latter can occur either at 
transmission or distribution level. Depending on the infrastructure level, the likelihood 
that a malfunction leads to an actual disruption (with consumers out of power) will vary, 
see Table 3. 


Table 3 : Infrastructure levels of malfunctions and their likelihood of causing disruptions 


Likelihood of 
disruption in case Intuition 
of malfunction 


Infrastructure Examples for causes for 


level malfunctions 


Tree falling on a distribution 
excog: (overhead) line, malfunction ; Distribution lines and cables are often 
a ofa transformer station, a but smaller the sole source of supply to a (small) 
storms damaging the area 
(overhead) lines 
Typically, a transmission line or cable 
Outage in an interconnector - especially an interconnector - is not 
Transmission in the transmission grid, low to high, larger | the only source of supply to an area. 
level snow or storms damaging area In case of a malfunction, electricity is 
overhead transmission lines supplied through other transmission 
cables or lines 


Source: Copenhagen Economics illustration 


Disruptions on transmission level occur much more seldom, but often have a large effect 
on the market. Several cases of transmission level events are covered as case studies 
towards the end of this chapter. 


In the following, we first evaluate a range of indicators for disruptions based on data 
provided by the Council of European Energy Regulators (CEER), before analysing 
disruptions based on reported single events. 


Two good indicators to gain an overall understanding of the extent of disruptions across 
Europe on an aggregated national level are: 


e the number of disruptions per year; this indicator is often referred to as System 
Average Interruption Frequency Index (SAIFI). 


e the total duration of disruptions per year; this indicator is often referred to as 
System Average Interruption Duration Index (SAIDI). 


To obtain a representative indicator that accounts for differences in the countries' sizes 
and number of customers, as well as for local disruptions that affected only parts of the 
country (which is typically the case), the number and duration of disruptions in a country 
must be weighted. The weighting methods applied across Europe differ, but show a 
strong trend towards weighing based on the number of customers. 23 out of the 28 EU 
Member States use a weighing method that is fully or partly based on the number of 
customers, see Table 4. Other weighting methods are based on the affected power or the 
number of affected transformers. 
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Table 4 : Weighting methods for disruption indicators across Europe 


weigh- 
ting 
method 


number 
of 
custo- 
mers 
power 
affec- x x x x x x x 
ted 
trans- 
former x x x 
stations 


other x x | x 


Note: No information available for Croatia. 
Source: CEER (2016) 6" CEER Benchmarking report on the quality of electricity and gas supply. 


The weighting means that the indicators can, for illustrative purposes, roughly be 
interpreted as per-person numbers. Important to bear in mind is that this interpretation is 
not perfectly consistent, as while following the same overall approach, the weighting 
methods differ slightly between countries. 


We focus on significant disruptions in this study. For the national-level indicators, we 
define significant disruptions as (1) "long" disruptions during which (2) consumers were 
out of power. With that definition, we follow the approach taken by CEER, which focuses 
on such "long" disruptions in its reporting. "Long" disruptions mean typically three 
minutes or longer; 24 of the European Member States use that duration threshold. The 
four Member States reporting disruptions in a different way are Denmark and the 
Netherlands, which report all disruptions of at least 1 minute and 5 seconds respectively, 
as well as Cyprus and Malta, which do not have a classification. 


For the period of 2010 to 2014, Romania has had the highest average number of 
significant disruptions with more than 6.5 disruptions per year, see Figure 40. 
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Figure 40 : Average number of disruptions (planned and forced) per customer, 2010- 
2014 
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Note: The graph shows a weighted indicator. The weighting is described in Table 4, and allows for an overall interpretation of 
the indicator as “per customer”. No data available for Belgium, Bulgaria, Cyprus and Slovakia. The average is a simple average 
across countries and is calculated based on the EU28-countries with available data. 


Source: Copenhagen Economics based on CEER data. 


Romania is followed by Croatia, Latvia, Malta and Poland, which still have on average 
more than 4 disruptions per year. Slovenia, Greece, Portugal, Italy and the Czech 
Republic have less than 4 disruptions per year, but are still above the EU average of 2.4. 
Luxembourg, the Netherlands, Denmark, Germany, the UK and Austria recorded the 
lowest average number of significant disruptions with a number smaller than 1. 


Looking at the minutes lost due to significant disruption events shows a similar overall 
pattern, see Figure 41. The minutes lost per year range from 17 (Luxembourg) to 844 
(Romania), with the (weighted)? EU average being 136 minutes lost per year over the 
period of 2010 to 2014. 


5^ Weighing based on the total electricity consumption of each country (2010-2014 average) 


81 


Figure 41 : Average minutes lost per year and customer due to significant disruptions 
(planned and forced), 2010-2014 
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Note: The graph shows a weighted indicator. The weighting is described in Table 4, and allows for an overall interpretation of 
the indicator as “per customer”. No data available for Belgium, Bulgaria, Cyprus. The EU-average is a weighted average across 
countries and is calculated based on the EU28-countries with available data. The weighting is based on total electricity 
consumption of each country. 


Source: Copenhagen Economics based on CEER data. 


Romania, Latvia, Croatia, Malta and Poland remain the top five, which means that they 
have both the most as well as the longest disruptions in the European comparison. Also, 
the countries reporting the lowest numbers remain the same. The Baltic countries Latvia, 
Estonia and Lithuania all have a worse position when looking at minutes lost instead of 
number of disruptions, which indicates that disruptions in those countries take longer 
than the EU average. In comparison, the Mediterranean countries Italy, Portugal and 
Spain have improved, indicating that significant disruptions in those countries are rather 
short in the European comparison. 


When making some simplifying assumptions^* and using (Member State level-) electricity 
consumption data, we can translate those average minutes lost per year into a rough 
estimate of how much electricity has not been supplied to consumers in the period of 
2010 to 2014. Roughly 550 to 850 GWh of electricity have not been supplied to 
consumers per year, see Figure 42. 


^ Applying the minutes lost per year per country to the average consumption per minute in this country, we 


implicitly assume that the disruption hit during a time (of day and year) of average consumption, as well as 
that it hit average consumers. The non-supplied electricity will be higher if the disruptions happened during 
peak demand times, and hit primarily consumers with above-average demand (and vice versa). Also, this 
calculation contains a small inconsistency arising from the fact that the Member States use slightly different 
weighting methods, as explained in the table above, and should therefore be used as a very rough estimate. 
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Figure 42 : Rough estimate for the non-supplied electricity due to disruptions in the EU, 
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Note: no data available for Belgium, Bulgaria, and Cyrpus 


Source: Copenhagen Economics based on CEER and Eurostat data. 


Those totals for the non-supplied electricity in Europe are the result of disruptions across 
all Member States. The countries with most non-supplied electricity through the years 
are, as one might expect looking at totals, larger countries like France, the UK, or Italy, 
see Figure 43. Compared to their size and energy consumption, Germany features rather 
low non-supplied electricity each year, while the numbers for Poland are comparably high. 


Figure 43 : Rough estimate for the non-supplied electricity due to disruptions per Member 
State, 2010-2014 
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Source: Copenhagen Economics based on CEER and Eurostat data. 
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The frequency of significant disruptions in Europe has shown a slight downward trend, see 
Figure 44. A simplified interpretation of the average indicator for frequency will be to say 
that an average European resident experienced 1.7 disruptions in 2010, and 1.4 
disruptions in 2014.*° The average duration per disruption in Europe decreases in the first 
two years of the period, from around 90 to around 80 minutes per disruption, but 
increases again back to around 90 in 2014. There is neither an upward nor a downward 
trend. 


45 This interpretation however should be used as a reference point for understanding only, as it contains a 


small inconsistency arising from the fact that the Member States use slightly different weighting methods, as 
explained in the table above. 


Final report 


Figure 44: Development of significant disruption events in terms of frequency and 
duration in the EU, 2010-2014 
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Note: The average across EU countries is weighted with total electricity consumption. No data available for Belgium, Bulgaria 
and Cyprus. For Slovakia, no data on duration is available, and the data on minutes of disruptions is incomplete and has been 
extrapolated for some years. The latter also holds for Luxembourg, where, however, only the year 2010 needed to be 
extrapolated from the rest. 


Source: Copenhagen Economics based on CEER data. 


For electricity consumers - both commercial and private - it makes a considerable 
difference whether a disruption is announced in advance, as typically done in case of 
planned disruptions, or whether it is forced disruption without notice. If informed in 
advance, consumers can plan accordingly and to a certain extent reduce the negative 
impact of a disruption. From a socio-economic perspective, forced outages are therefore 
more problematic than planned ones. 


Forced disruptions dominate over planned ones in Europe, see Figure 45. Measured in 
number of significant disruptions, 4 out of 5 are forced ones; measured in minutes lost, 
two thirds of all disruptions are forced ones. This difference between the measurements 
indicates that forced disruptions are typically shorter in duration than planned ones. 
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Figure 45 : Share of planned vs. forced significant disruptions in Europe 2010-2014 


number of minutes 
significant lost, 
disruptions, : EU total 
EU total planned; 


0, 
13% planned 


; 33% 


Note: No data available for Belgium, Bulgaria, Cyprus and Slovakia. For Luxembourg 2010, averages of 2011-2014 have been 
used. 


Source: Copenhagen Economics based on CEER data. 


The average number of significant forced disruptions varies from 5.5 for Romania to 0.3 
for Luxembourg, see Figure 46. The European average is at 1.9. 


Figure 46 : Average number of significant forced disruptions per customer, 2010-2014 
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Note: The graph shows a weighted indicator. The weighting is described in Table 4, and allows for an overall interpretation of 
the indicator as “per customer”. No data available for Bulgaria, Cyprus and Slovakia. The average is a simple average across 
countries calculated based on the EU28-countries with available data. 


Source: Copenhagen Economics based on CEER data. 


Looking at the minutes lost per year due to significant forced disruptions shows a similar 
pattern, see Figure 47. Consumers in Romania are on average 563 minutes each year out 
of power due to forced disruptions; that is the highest value within the EU. Denmark 
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features the lowest value with 15 minutes on average. The European average is at 175 
minutes lost. 


Figure 47 : Average minutes lost per year per customer due to significant forced 
disruptions, 2010-2014 
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Note: The graph shows a weighted indicator. The weighting is described in Table 4, and allows for an overall interpretation of 
the indicator as "per customer". No data available for Bulgaria and Cyprus. The average is a simple average across countries 
and is calculated based on the EU28-countries with available data. 


Source: Copenhagen Economics based on CEER data. 


The average duration of forced disruptions in Europe is 84 minutes, see Figure 48. 


Figure 48 : Average duration per significant forced disruption, 2010-2014 
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Note: No data available for Bulgaria, Cyprus and Slovakia. The average is a simple average across countries and is calculated 
based on the EU28-countries with available data. 


Source: Copenhagen Economics based on CEER data. 
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In the following, we will move away from overall indicators and will analyse disruptions 
based on reported single events. The data sources for the following figures are the 
ENTSO-E Monthly Reports, as well as a survey that we sent to TSOs, DSOs and NRAs in 
the 28 Member States. As not all countries reported all of their disruptions, using this 
data to draw conclusions on absolute magnitudes would most probably mean that we 
significantly underestimated disruptions in Europe. However, assuming that there is no 
structural bias, i.e. that the collected data is a representative subset of the EU, we can 
draw conclusions on relative magnitudes and shares. 


The dataset contains 388 observations for 2010 to 2016 from 22 Member States in total. 
Due to limited data, we decided not to exclude any further observations, and report all 
disruption events that we obtained data for. The fact that we defined a « disruption » as 
an event where consumers are out of power means that we automatically focus on those 
events that mattered for society; we therewith apply an implicit filter for significance on 
all events. 


Disruptions can occur from different causes. Of all the single disruption events reported, 
most are due to false operation, failure of equipment or material damage (40%), followed 
by severe weather conditions and natural hazards (33%), see Figure 49. Only 2% of the 
disruptions were reportedly due to human errors. 


Figure 49 : The reasons for disruption events 
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Source: ENTSO-E Monthly Statistics Reports as well as a survey to European TSOs, DSOs and NRAs 


In our survey, we investigated in particular to which extent disruptions are caused by 
outages, or have been linked to malicious attacks. To the question whether any 
disruptions were caused by outages, 13 of the 18 countries that responded stated that 
none of their disruption events has been linked to an outage, see Table 5. 
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Table 5 : Disruptions caused by outages - the Member States’ responses 


Have any 
disruptions 
been caused 
by outages? 


Member States’ responses 


13 countries: Austria, Belgium, Czech Republic, Estonia, Finland, Italy, Latvia, Lithuania, 


No Luxembourg, Portugal, Slovakia, Slovenia, Spain 


5 countries: 


e Ireland reports that two supply disruptions since 2010 were due to under frequency 
disturbances caused by the tripping of large generators 


e Malta reports that in total, 69 supply disruptions have been linked to outages since 
2010, with a decreasing tendency in the recent years. The number of disruptions linked 
to outages was in each of the years since 2010: 13 (2010), 10 (2011), 13 (2012), 12 
(2013), 16 (2014), 5 (2015) and 0 (2016). 


Yes e Poland reports that typically, disruptions in the country are not linked to outages, but it 
does happen irregularly in one of Poland's regions, where there are challenges 
regarding energy generation by renewable sources and the distribution in the grid. 


e Romania reports that it is rare that disruptions are caused by outages, but it happened 
in one case in 2016 when a generation plant at TSO level had an outage. 

e The UK reports several cases where failures of supply from an electricity generation 
company led to disruptions; The number of such disruptions since 2010 was: 25 
(2010/11), 45 (2011/12), 14 (2012/13), 7 (2013/14), 8 (2014/15), and 1 (2015/16). 


9 countries: Bulgaria, Croatia, Cyprus, Denmark, France, Germany, Greece, Hungary, 


n/a Sweden 


Source: Copenhagen Economics illustration based on survey results 


To the question whether any disruptions were caused by malicious attacks, the clear 
majority reported that this had not been the case. France, Italy and Poland are the 
exemption and report cases of theft, see Table 6: 


Table 6 : Disruptions caused by malicious attacks - the Member States’ responses 


Have any 
disruptions 
been caused 
by outages? 


Member States’ responses 


17 countries: Austria, Belgium, Croatia, Czech Republic, Denmark, Estonia, Finland, 
No Ireland, Latvia, Lithuania, Malta, Portugal, Romania, Slovakia, Slovenia, Spain, United 
Kingdom 


3 countries: 


e France reports that the French TSO had only one significant supply disruption in 2016, 
where a theft of cable/line caused a busbar fault in a VHV/HV transformer station. The 
disruption had a magnitude of 121 MWh of non-supplied electricity. 


Yes e Italy reports that there have been one or more cases of theft. 
e Poland reports that typically, disruptions in the country are not linked to malicious 
attacks, but energy infrastructure thefts do happen and can lead to disruption events. 


That was the case for the following number of events: 2010 (408), 2011 (529), 2012 
(553), 2014 (1956), 2015 (840) and 2016 (858). 


8 countries: Bulgaria, Cyprus, Germany, Greece, Hungary, Luxembourg, the Netherlands, 


n/a Sweden 


Source: Copenhagen Economics illustration based on survey results 


89 


The largest disruptions 2010-2016: TOP20 


The reported disruptions in Europe show a wide range in their magnitude in terms of non- 
supplied electricity to consumers. The disruptions range from less than 1 MWh of non- 
supplied electricity to more than 21,000 MWh. In fact, the largest 20 disruptions did 
cause almost 70% of the total non-supplied electricity in 2010-2016, see Figure 50. 


Figure 50 : Merit-order-graph of all reported disruptions ranked after market impact 
(non-supplied electricity, MWh) 


non-supplied 
electricity (MWh) 
25.000 
20.000 
15.000 


10.000 


5.000 


observation 


187 


Source: ENTSO-E Monthly statistics reports and survey. 


A closer look at the 20 largest disruptions in terms of non-supplied electricity to 
consumers since 2010 supports the conclusion that most disruptions are due to either 
weather conditions or failure of equipment. None of the largest disruptions has been 
caused by an outage or by a malicious attack. Further details on the largest events are 
provided in Table 7. 
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Table 7 : The 20 largest disruption events in Europe since 2010 
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supplied electricity to 
consumers (MWh) 


Amount of non- 


Cause and details 


Cause: self-detonation of containers of explosives in a naval base resulting in the destruction 


1 Cyprus 2011 7 21,575 of the whole Vasilikos Power Station (biggest power station of the country). iE 
Cause: weather condition (storm). 

2 Portugal ante 1 oe 5,449 Affected infrastructure element: transformer stations and overhead lines. survey 

3 Germany 2010 | 11 11 5,500 Cause: False operation, failure of equipment or material damage ENTSO-E 

4 Germany 2014 9 4 3,112 Cause: tripped generator due to an incident originated in the 380 kV substation Uentrop ENTSO-E 
(failure in the combined instrument transformer Sauerland Nord line) 

: Cause: weather conditions, natural hazards 

5 create ie 2 2,794 551,000 consumers where affected. suryey 
Cause: weather conditions, natural hazards. 
Series of material failure on measuring transformers on numerous VHV/HV/MV substations, 

6 FEES 2029 z T 2,337 during a heat wave. Brownout of 36 delivery points and load shedding on 21 delivery points. Survey. 
Affected infrastructure element: transformer station. 
Cause: weather conditions, natural hazards 

7 Portugal 2040 2 8 2,175 Affected infrastructure element: transformer stations and overhead lines. sane 

8 Germany 2010 3 3 2,135 Cause: False operation, failure of equipment or material damage ENTSO-E 
Cause: weather conditions, natural hazards. 

3 Italy | aU 1,783 Affected infrastructure element: line Survey 
Cause: weather conditions, natural hazards. 

10 France 2012 3 96 1,751 Series of incidents on overhead lines during an episode of sticky snow. Brownout of 20 survey 
delivery points. 
Affected infrastructure element: cable/line. 

, Cause: weather conditions, natural hazards. 
im Croatia 2014 E 1,500 15,000 consumers were affected Survey 
12 Germany 2011 4 1 1,167 Cause: fire in the substation Bürstadt of the transmission system operator. ENTSO-E 
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Amount of non- 


supplied electricity to 
consumers (MWh) 


Cause and details 


Cause: weather conditions (icing of overhead line) 


En Slovenia 2014 =| -200 1088 Affected infrastructure element: interconnector at transmission level. survey 
Cause: weather conditions (storm) 

14 France 2010 6 72 1,074 Affected infrastructure element: transformer station. Flooding and mudslides in a VHV/HV survey 
substation caused brownouts on 12 delivery points and load shedding on 19 delivery points. 
Cause: weather conditions (snow) 

13 Portugal an 2 4B 1,020 Affected infrastructure element: transformer stations and overhead lines. Survey 
Cause: weather conditions, natural hazards. 

16 Italy 2015 2 BET Affected infrastructure element: line survey 

17 Germany 2010 6 3 942 Cause: weather conditions, natural hazards. ENTSO-E 
Cause: weather conditions (storm) 

18 Portugal eode Ede 213 Affected infrastructure element: transformer stations and overhead lines. Survey 

19 Germany 2014 9 1 916 Cause: tripped generator due to an incident originated in the 380 kV substation Uentrop ENTSO-E 
(failure in the combined instrument transformer Sauerland Nord line) 

20 Italy 2015 3 892 Cause: weather conditions (snow) survey 


Note: Bulgaria, Cyprus and Greece did not answer to the survey. 


Source: ENTSO-E Monthly statistics reports, survey, literature research 
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Disruptions in the transmission grid happen more rarely but with a larger effect 


In the above description of the largest disruptions in 2010-2016, it is interesting to see 
that very few of the disruptions are related to transmission grid failures. History has 
shown, that while transmission grid failures occurs quite rarely, they have potential to 
lead to massive disruptions both in terms of length of the duration and number of 
consumers affected. In this section, we look more closely at a number of different cases 
of transmission grid failure, and how they affected consumers’ access to power in the 
grid. We will in the following distinguish between two types of disruptions: 


1. Cases of an actual disruption, meaning that consumers are out of power due to a 
problem in the transmission grid, for example a failure in an overhead line. The 
direct impact on consumers will be large, as there is no access to consume 
electricity. This will not however necessarily translate into higher electricity prices 
in the region, as there simply is no infrastructure to facilitate market transactions 
(prices in neighbouring regions could be affected). 


2. Cases of interconnector disruptions, sometimes combined with further problems in 
the transmission grid, which lead to stress in the grid, but not to an actual 
disruption in the sense that consumers are out of power. Such cases will typically 
not cause disruptions, but can increase electricity prices. 


The following five case studies illustrate which shapes disruptions or problems in the 
transmission grid can take. The first three examples from Italy, Germany and Denmark 
are disruptions of type 1, where consumers were out of power. The latter two examples 
from Sweden, France and the UK are type 2-cases. 


On the night of Sunday 28 September 2003, a sequence of events caused a blackout in 
almost all of Italy as well as parts of Switzerland that affected around 56 million people, 
see Box 1. 


Box 1: Blackout in Italy in 2003 with around 56 million people out of power 
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The causes 

The blackout was caused by a sequence of events triggered by a trip of the Swiss line Mettlen-Lavorgo 
caused by a tree catching fire at 03:01am. The automatic and manual attempts to re-close the line were 
unsuccessful. Meanwhile, other lines had taken over the load of the tripped line, but the other Swiss line Sils- 
Soazza nearby was overloaded, which is normally acceptable for up to 15 minutes. At 03:11am, the Swiss 
coordination centre of ETRANS called the control centre of the Italian TSO GRTN (now Terna since 2005) to 
request countermeasures within the Italian system, namely the reduction of Italian electricity import, which 
was in excess by 300 MW compared to the agreed schedule. This import reduction was effective at 03:21am, 
but, together with some internal countermeasures taken within the Swiss system, it turned out insufficient to 
relieve the overloads. At 03:25am, the line Sils-Soazza also tripped after a tree catching fire, probably in 
contact with the line being overheated due to the overload. Having lost two important lines, the overloads on 
the remaining lines in the area became intolerable, and the instability of power and voltage caused an 
almost simultaneous and automatic trip of the remaining interconnectors towards Italy. As a result, the Italian 
system was isolated from the European network about 12 seconds after the loss of the line Sils-Soazza. 

2 minutes and 30 seconds after the separation of the country, the frequency drop in Italy led to a collapse 
of the system, causing a blackout. 


The restoration process in Italy started immediately after the blackout. Nearly all of the northern part of Italy 
was energised before 08:00am, the central part around 12:00am and the remaining parts of mainland Italy at 
17:00pm. Sicily was fully energised at 21:40pm. The main reason for the difference in timings was the failure of 
several hydro plants in southern Italy to black-start. 


The impact 

The result was an electricity blackout in all of Italy (except the islands of Sardinia and Elba) for approximately 
12 hours and a part of Switzerland for 3 hours, affecting a total of around 56 million people for a total energy 
not delivered of about 180 GWh (IEEE 2007). 


Source: UCTE (2004) final report of the investigation committee on the 28 September 2003 blackout in Italy; Corsi, S. and 
Sabelli, C. (2004) General blackout in Italy on Sunday 28 September 2003; IEEE (2007) Blackout experiences and lessons, best 
practices for system dynamic performance, and the role of new technologies. 


In November 2006, transmission system disturbances occurred in Germany, France 
and Italy. A high voltage line in Germany had to be switched off to let a ship pass 
underneath, which led to overloading of lines and eventually a disruption affecting more 
than 15 million households, see Box 2. 


Box 2: Disruptions on the transmission level in Germany, France and Italy in 2006 
affecting 15 million households 
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The causes 

On the 4" of November 2006, system disturbances occurred in Germany, France and Italy. The fault originated 
from Northern Germany, from the control area of the TSO Eon Netz (TenneT since 2009). A high voltage line 
had to be switched off to let a ship pass underneath. After manual disconnection of the line to let the ship pass, 
the N-1 criterion was not fulfilled in the E.ON Netz grid and on some of its tie-lines to the neighbouring 
TSOs. The incident demonstrated insufficient inter-TSO co-ordination, as a change of time had occurred for 
switching-off the line but E.ON Netz communicated the change very late to the other involved TSOs (RWE TSO 
and TenneT). The switching was therefore insufficiently prepared to ensure the secure operation. This led to 
overloading of lines and finally to splitting of the Union of Co-ordination of Electricity Transmission (UCTE, now 
ENTSO-E since 2009) network into three zones: West, North-East and South-East. The Western zone lacked 
power and the Eastern zone had too much power. The power imbalance in the Western area induced a severe 
frequency drop that caused a disruption in several countries, hitting France and Germany hardest. 

In both under-frequency areas (West and South-East), sufficient generation reserves and load shedding allowed 
to restore the normal frequency within about 20 minutes. In the over-frequency area (North-East), the lack of 
sufficient control over generation units contributed to the deterioration of system conditions in this area (long 
lasting over-frequency with severe overloading on high-voltage transmission lines). Generally, the uncontrolled 
operation of dispersed generation (mainly wind and combined-heat-and-power) during the disturbance 
complicated the process of re-establishing normal system conditions. Full resynchronization of the UCTE system 
was completed 38 minutes after the splitting and the TSOs were able to re-establish a normal situation in all 
European countries in less than 2 hours. 


The impact 

The result was a blackout affecting more than 15 million European households. The most affected area was 
France where 5 million customers were cut-off. In Germany millions of customers were also affected, while in 
Belgium, Netherlands, Italy and Spain some hundreds of thousands of customers were without electricity. 


Source: UCTE, Final Report - System Disturbance on 4 November 2006. 


In the years 2002 and 2003, there have been two disruptions at the transmission grid in 
Denmark that caused significant blackouts, in the 2002 case for about 20% of the 
population, see Box 3. 


Box 3: Disruptions in the Danish transmission grid in 2002 and 2003 hit millions of 
consumers 
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The causes 

On 28 December 2002, two technical failures happened independently of each other in the transmission 
grid in Western Denmark, namely an error in the relay of one of the main cables/lines between Kassg (a town 
close to the German border) and Tjele (a town further north in Jutland, close to Viborg). 


Another large-scale disruption hit Eastern Denmark and Southern Sweden on 23 September 2003 at 12.37 
noon. The primary cause was technical infrastructure failure at transmission level in Southern Sweden, 
which caused unavailabilities at several units of a nuclear power plant in the region. Just before, there had been 
an outage of a unit of the different nuclear power plant. The situation resulted in a voltage breakdown in 
Southern Sweden as well as Eastern Denmark, which is connected to Southern Sweden through an 
interconnector. 


The impact 

On 28 December 2002 in the early morning (6.45am), about one million people in North-Western Denmark - 
that is about 209/o of the country's population - were out of power for up to three hours. Power supply 
was up again for the first consumers at 7.35am and the last ones at 9.50am. 


On 23 September 2003, the disruption took several hours. In Denmark, power supply was up again after a 
little more than an hour for the first consumers (1.47pm), and only after more than six hours (7.05pm) for the 
last ones. 


Source: Energinet (2017) Report on power supply security; Danish Energy Agency (2015) Power supply security in Denmark; 
TV2 News (2003) 10 years of blackouts in Denmark 


The following two examples show cases of problems in the transmission grid that did not 
lead to blackouts. 


In December 2010 in Sweden, unavailabilities of several interconnectors combined with 
reduced power supply and high power demand resulted in an extremely tight market 
situation. Prices increased significantly, and several price peaks with extraordinary high 
prices occurred, see Box 4. However, disruptions were avoided, and power was delivered 
reliably to all customers at any time. 


Box 4: High prices, but no disruptions as a consequence of a situation resembling a 
"perfect storm" in Sweden in December 2010 
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The causes 

Several interconnectors were out of order (Swepol Link to Poland of 600 MW capacity, connection to 
Western Denmark of 740 MW capacity) or had a reduced capacity (interconnector from Germany to Eastern 
Denmark, connection to Southern Norway), which limited Sweden’s possibilities to import power. 

In addition to those interconnector unavailabilities, three other independent circumstances led to a reduced 
power supply. A series of Swedish nuclear power plants were running at a 20% reduced capacity 
(2,000 MWh less than usual) due to operational problems, while nuclear power accounts for about 40% of the 
annual Swedish power generation. The level of Swedish water reservoirs was 30% lower (7,500 GWh 
less) compared to the typical level at that time of the year, while hydropower accounts for about 50% of the 
annual electricity production. The level of Norwegian water reservoirs was also 30% lower than the usual 
level and as the Nordic electricity markets are closely linked, this affects the Swedish power market 
considerably. 


The reduced 

S power supply 
ais coincided with 
higher-than- 

2000 usual demand 
for electricity 

1500 - due to extremely 
low temperatures. 
The December 

1000 - | 
was the coldest in 

Sweden since 

x 1987, with 


temperatures 5- 
15 degrees lower 


123 4 5 6 7 8 9 0 t1 2 B 4 5 1 17 18 19 20 21 22 23 24 25 26 27 28 29 30 pi than usually at 
mum dec- 10 —— dec-08 that time of the 
year. The 


increased heating meant that power consumption was 4,000-6,000 MWh higher than usually. 


The impact 

This combination of events resulted in significantly higher electricity prices at the spot market than usually in 
December. The figure below illustrates the two main price effects: a higher general price level and more 
frequent price peaks of an larger magnitude. 


Important to note, only some consumers were actually hit by those high electricity prices. Consumers on fixed 
price contract paid a similar price per kWh as in 2008, even slightly lower. Consumers on a contract with a 
flexible price experienced price increases of 5096, from around 63 to 97 Óre per kWh. 
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Source: Copenhagen Economics for E.ON Sweden (2010) That’s how it works in the power market. Analysis of the price peaks in 
December 2010. 


An interconnector failure in November 2016 between France and the UK was expected 
to have a considerable impact on the British market. However, disruptions were avoided 
and the British TSO reported “no significant impact on electricity prices” from this event, 
see Box 5. 


Box 5: Interconnector failure between France and the UK in 2016 


The causes 

The Interconnexion France-Angleterre (IFA) link between Folkestone and Calais is Britain's biggest 
interconnector, allowing it to import up to 2 gigawatts of power from the continent when UK supplies run low. 
During a storm (Storm Angus), in the morning of the 20th of November 2016, four of eight cables have 
been damaged, putting 1GW of capacity out of action until the end of February 2017. The cables have been 
damaged when a ship dropped urgently an anchor during the storm. The anchor hit a French pair of 
cables and then bounced over a British pair before crashing into another French pair. 


The faults in the cables were about three miles off the coast in water depths of about 65 ft. The cables were 
heavily armoured and buried within the seabed, and damage on such a scale was unexpected. The 
interconnector typically supplies about 5% of the UK’s electricity and brings a degree of flexibility to the UK 
energy system. Moreover, the impact of the outage was complicated by the fact that a series of nuclear reactor 
safety shutdowns in France had significantly reduced French power supplies and increased power prices. 


The impact 

Analysts at Barclays were forecasting at the time of the incident that the outage was likely to lead to “increased 
volatility and higher UK power prices over January and February 2017 - especially during peak demand 
periods”. However, the British TSO National Grid reports that there has not been an impact on prices, and 
that the overall impact on the market and their business was very small and almost negligible. 


Source: Interview with National Grid; 4coofshore (2017) Presentation of the Interconnection France - UK 2000 (IFA2000) 
Interconnector; The Telegraph (2016) Winter power crunch fears as UK-France cables severed during storm; The Guardian 
(2017) All it took was an anchor, how a storm took down half the UK's electricity link to France. 


4.2.3 Conclusions 


Power supply is very reliable in the EU. For an average EU country, power consumers will 
on average have access to power in 99.948% of the time (measured in 2014). This 
means than an average power consumer in Europe will face about 1 to 2 disruption 
events longer than 3 minutes each year. From 2010 this number has fallen from 1.7 to 
1.4 in 2014. Reliability varies across individual Member States with Luxembourg, 
Denmark, Germany and the Netherlands at the high end with around 20 minutes to half 


46 “Longer than 3 minutes” is the most commonly used threshold across the EU, with 24 of the 28 Member 


States using it. The four Member States reporting disruptions in a different way are Denmark and the 
Netherlands, which report all disruptions of at least 1 minute and 5 seconds respectively, as well as Cyprus 
and Malta, which do not have a classification. 
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an hour of disruptions per customer per year, and Romania, Latvia and Croatia in the low 
end with about 10 to 14 hours of disruptions per customer per year. 


By far the majority of disruptions occur at the distribution level. Disruptions at 
transmission level are rare, but if they happen, they often have a considerable impact - if 
they lead to a blackout - on hundreds of thousands or even millions of consumers 
directly, as examples from Italy, Germany, France and Denmark show. For the 
transmission cables and lines between countries (interconnectors), failure alone very 
rarely gives rise to significant distress to the power system let alone actual disruptions to 
consumption. Instead, it leaves the national/regional power market more vulnerable to 
additional simultaneous failures in e.g. local generation assets, and may increase the 
power prices in the countries depending on the local supply base. An illustrative example 
from Sweden in 2010 showed that significant price increases can indeed happen, but it 
typically requires similar significant events in parallel such as generation assets out for 
maintenance. Even the combination of multiple significant events in Sweden in 2010 did 
not give rise to any brownouts. 


When a power disruption happens, it is either due to a malfunction in the grid 
infrastructure either at distribution level or transmission level, or due to a lack of 
generated power. While the vast majority of the disruptions occur due to problems in the 
grid such as weather-related breakdown of overhead lines or transformer station 
malfunctions, there are incidents where a lack of power generation to meet supply has led 
to disruptions. These incidents are quite rare. When the disruptions are related to 
infrastructure, it is almost always caused in the distribution grid. All 9 countries 
answering this particular question stated that disruptions typically happen at DSO level, 
mostly at electric lines. 


About a third of the minutes of disruptions are planned disruptions e.g. for maintenance 
or construction purposes, and two thirds are unplanned disruptions. The primary reason 
for the unplanned disruptions are so-called false operation, failure of equipment or 
material damage (40%) followed by severe weather conditions and natural hazards 
(30%). We find very little evidence that disruptions have been due to a malicious attack. 
17 countries report that no disruption was caused by a malicious attack, only Poland, 
France and Italy report single cases of thefts of infrastructure elements that have caused 
disruptions. 


We estimate that for EU in total approximately 600 to 850 GWh of electricity have not 
been supplied to consumers per year in the years 2010-2014. This constitutes both 
planned and unplanned disruptions. 


4.3 Output 5 - The use of voluntary demand curtailment 


4.3.1 Objective and approach 


Output 5 aims at gathering information on voluntary demand curtailments in case of 
significant electricity supply disruptions, and to establish estimations of the electricity 
supply margins thus created. The overall objective of this output is to provide an 
understanding to which extent voluntary demand curtailment has been used as an 
effective solution today (and/or could be in the future) in times of severe stress in the 
energy system. 
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The information was collected in two steps. First, we carried out desk research through 
European sources (e.g. ENTSO-E, JRC, Smart Energy Demand Coalition) and national 
sources (reports and websites of the NRAs, TSOs and DSOs)" to identify the voluntary 
demand curtailment mechanisms in place, the frequency of their activation in cases of 
significant supply disruptions and the margin supply thus created. To complement the 
information, we launched a survey with the national TSOs and where relevant, DSOs, and 
received answers from 14 Member States. Where relevant, we followed up with additional 
questions by email and phone with the TSOs involved. 


4.3.2 Findings 


4.3.2.1 Overall findings and conclusions 


Traditionally, balancing the power system to ensure the right frequency has been a 
question of regulating the supply side, that is ramping power generators up or down. 
However, the demand side is increasingly seen as a potential resource for balancing 
purposes due to several reasons. Firstly due to technological developments making 
demand response easier to use for balancing purposes, and secondly due to the expected 
increase in large-scale demand assets in individual households such as electrical vehicles 
and heat pumps. 


Demand side flexibility is viewed upon with great potential as it is indeed the current 
inflexibility that gives rise to the discussions about reserve availability, capacity payments 
etc. If it could be ensured that consumers were easily and cheaply available to 
communicate and respond flexibly to price variation, many of the issues related to energy 
system balancing would be solved. 


However, there are still a number of challenges left with respect to utilising the demand 
side as a balancing asset such as 1) it is not deemed as reliable as supply assets, 2) it 
cannot typically be available for as long as supply assets, 3) the demand assets are 
substantially smaller in size and therefore require significant coordination efforts to 
mobilise sizeable capacity and 4) it is relatively expensive to mobilise especially 
household assets as it requires investments in e.g. smart metering equipment, new 
business models and new regulation and market design models. 


Nonetheless, the demand side seems to play at least some role in many European 
Member States. In particular the UK seems to be on the forefront of incentivising demand 
side response and has by far the most developed market for demand-side aggregators. 


Demand response can broadly be categorised in two groups: 


- Implicit demand response (or "price-based") whereby consumers choose to be 
exposed to time-varying electricity prices that partly reflect the value or cost of 


47 The full list of literature is available in Annex 8. 
^5 JRC, Demand Response Status in EU Member States, 2016. http://publications.jrc.ec.europa.eu/repository/ 
bitstream/JRC101191/ldna27998enn.pdf 
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electricity and/or transportation in different time periods. These prices are part of 
their supply contract. 

- Explicit demand response whereby demand competes directly with supply in the 
wholesale, balancing and ancillary services markets through the services of 
aggregators or single large consumers. Consumers receive direct payments or 
better conditions to change their consumption upon request, either individually or 
by contracting with an aggregator which can be either a third-party aggregator or 
the customer's retailer. 


Voluntary demand curtailment mechanisms belong to the category of explicit demand 
response and consist in medium/long-term agreements where consumers allow TSOs or 
DSOs to curtail them upon request in cases of impending significant supply disruption 
events. Voluntary demand curtailment mechanisms are activated in emergency cases 
where electricity demand exceeds supply and where consumers are inevitably going to be 
cut out of power. 


It has generally been very difficult to find information on the extent to which demand side 
assets have contributed as an effective tool to avoid brownouts, as this is rarely recorded 
and/or published by TSOs. To the extent that demand side assets contribute to balancing 
mechanisms and reserve pools, we expect that these have been called upon in times of 
severe distress, but that the impact has been quite small compared to the activation of 
generation assets. 


However, some countries also engage in specific demand curtailment programmes used in 
emergency circumstances. These programmes allow for a more controlled disconnection 
of consumers before an actual brownout that disconnects an entire geographic area. 12 
countries seem to have such voluntary demand curtailment mechanisms in place and in 
about 8 out of these 12, we have found evidence that these demand side programmes 
have been activated, see map. 


Figure 51 : Activation of voluntary demand curtailment mechanisms 2010-2016 
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M Evidence of activation 
M No evidence of activation 


However, the capacity represented by these voluntary demand curtailment mechanisms 
represents on average only 2% of the total electricity capacity in the respective Member 
States, while in the country where the volume curtailed was available, this amounted only 
to 0.0013% of the total electricity consumption in these countries. 


4.3.2.2 Detailed findings 


The mapping of demand response programmes across the Member States allowed to 
identify 12 countries where voluntary demand curtailment mechanisms exist. For some of 
them, the information regarding the frequency of their activation and the supply margin 
created could also be found, while in some other countries it was not publicly available. 


The table below presents the volume contracted under voluntary demand curtailment 
mechanisms as a share of the total electricity capacity in the respective Member States in 
2016. In half of the countries, this share is below 1% while the highest share is recorded 
in Portugal with such mechanisms representing 10.2% of installed capacity. Information 
on the volume of demand response curtailed under these mechanisms was only available 
in Belgium but this volume represented only 0.0013% of the total electricity consumption 
in 2015. 


Table 8: Volume contracted and curtailed compared to total capacity and consumption in 
2016 
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Member Voluntary Volume Share Volume Share 
State demand contracted volume curtailed volume 
curtailment (MW) contracted/ (MWh) curtailed/ 


mechanism total total 

electricy electricity 

capacity consumption 

(%) (%) 
Interruptible 199 0.9 1,070* 0.0013* 
contract 
programme 
Strategic reserve | 97 0.5 0 0 
Finland Peak load | 299 1.9 0 0 
reserves 
France NEBEF 2,100 1.6 N/A N/A 
Interruptible 1,600 1.2 N/A N/A 
contracts 
Germany Interruptible load | 1,500 0.8 N/A N/A 
programme 
Great Short-Term 78 0.1 N/A N/A 
Britain Operating 
Reserve 
Demand-Side 515 0.7 N/A N/A 
Balancing 
Reserve 
Greece Demand Side | 611 3.7 N/A N/A 
interruptible 
programs 
Ireland Short-Term 45** 0.6** N/A*** N/A*** 
Active Response 
Italy Interruptible 4,061* 3.4* N/A N/A 
contracts 
programme 
Netherlands | Emergency 350 1.2 N/A N/A 
capacity 
Poland Emergency 200 0.5 0 0 
Demand 
Response 
Programme 
Portugal Interruptible 2,000 10.2 0 0 
contracts 
programme 
Spain Interruptible load | 2,890 2.7 N/A N/A 
programme 
Note: * 2015 figures as 2016 figures were not available; ** 2014 figures as 2015 and 2016 figures were not available; 
*** In Ireland in 2016 a total of 138 MW was curtailed under the STAR programme. 


Belgium 


e Belgium 
The Interruptible contract programme (ICH), whereby large consumers receive 
payment from the TSO for the availability of reserve and its activation, offered a capacity 
of 261MW for 2014 and 2015, 199 MW for 2016, and 200MW for 2017. In 2016, this 
represented only 0.9% of the total electricity capacity of Belgium.*® The frequency of the 
Interruptible contracts activation, and the supply margin thus created is presented in the 
table below. 


Table 9 : Activation of the Interruptible contract programme (ICH) 


^" FEBEG, Electricity Statistics. Available at: https://www.febeg.be/fr/statistiques-electricite 
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Number of | Volume Share of Total number Average Average 


Interruptions curtailed total of hours of number of volume 
(MWh) consumption interruption curtailed 

(CO) (MWh) 
2010 9 4,752 0.0055 42 4.7 113 
2011 4 1,884 0.0023 11 2.7 176 
2012 2 564 0.0007 4 2.1 135 
2013 3 572 0.0007 3 1.0 200 
2014 1 1,163 0.0014 5 4.5 258 
2015 2 1,070 0.0013 5 2.3 238 


Source: CREG, Elia, FEBEG 
Note: For one event, several clients may have been interrupted. 


The activation of this programme remains limited, as well as the volume of energy 
curtailed compared to the total consumption (0.002% in average) and the duration of the 
curtailments. This is due to the fact that this programme is only used in case of severe 
disruptions and that the contracts only allow few activations per year. It should be noted 
that the NRA has requested the TSO to activate the contracts minimum once a year to 
maintain the experience of the service providers. 


Demand Response also represents about one tenth of the capacity involved in the 
Strategic Reserve, which represented 0.5% of the total electricity capacity of Belgium in 
2016, but to date, the Strategic Reserve has not been activated. The TSO deems the use 
of demand response as effective as generation participation in the Strategic Reserve 
mechanism. 


e Finland 


Consumers can participate in the Strategic Reserve mechanism since 2013, which 
consists in contracts between the TSO and the largest industrial consumers to provide 
peak load reserves. Demand response in peak load reserve represents 10MW for a total 
of 299MW in the 2015-2017 period, and 22MW over 729MW for 2017-2020. In 2016, this 
demand response represented only 1.9% of the total electricity production capacity.°° The 
use of peak load reserve capacity is very rare, the last time that the reserve was 
activated was during the winter 2009- 2010 (when demand response was not 
participating yet). 


e France 


Demand response can participate in the NEBEF ("Notification d'Échange de Blocs 
d'Effacement") since 2014, whereby curtailed load can bid as energy directly into the 
wholesale electricity market. In addition, residential consumers can participate in the 
NEBEF by receiving a premium for the consumption reductions that they provide (so 
called "effacement résidentiel diffus"). This premium is financed through the tax included 
in the electricity tariffs, however, this premium is under question as a possible subsidy 
and its future is unclear at this stage. Overall the volume of demand response load 
contracted under the NEBEF amounted to 850 MW in 2014, 1200 to 1800 MW in 2015, 
2100 MW in 2016, 750 to 1400 MW in 2017. In 2016, the demand response share of the 
NEBEF represented 1.6% of the total electricity production capacity in France.?! The 


50 Finland Energy Authority, 2017 National report to ACER. Available at: 
https://www.ceer.eu/documents/104400/5988265/C17 NR Finland-EN.pdf/b1048901-ce81-7586-4a9f- 
5f9fdb4ce5b8 

5! RTE, Bilan Electrique français 2016. Available at: http://www.rte- 


france.com/sites/default/files/2016_bilan_electrique_synthese. pdf 
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volume activated under the NEBEF was 310 MWh in 2014, 1522 MWh in 2015 and 10313 
MWh in 2016, however the part of demand response in the NEBEF is linked to market 
prices thus it is hard to link it to specific disruption events. According to the TSO, 
consumer participation in the NEBEF is effectively contributing to reducing the risk of 
brownouts and blackouts. 


Since 2013, direct interruptible contracts exist between the TSO and electricity- 
intensive consumers. The maximum contracted volume of interruptible contracts was 
400MW for 2013, 2014 and 2015, and 1600MW for 2016 and 2017. In 2016, interruptible 
contracts represented 1.2% of the total electricity production capacity in France. The 
frequency of their activation and the supply margin created are not publicly available. 


e Germany 


Large consumption units can participate in the interruptible load programme since 
2013. Between 2013 and 2015, the TSO had a monthly auction for 1000 MW of 
immediately interruptible loads and 1.000 MW of quickly interruptible loads (but not even 
half was tendered due to the entry barriers). Since 2016, there is a weekly auction of 750 
MW of immediately interruptible loads, and 750 MW of quickly interruptible loads, which 
overall represents 0.8% of the total electricity generation capacity in Germany.?? The 
frequency of activation of interruptible loads and the supply margin created were not 
publicly available. 


e Great Britain 


Consumers can participate in the Short-Term Operating Reserve (STOR), which 
spurred demand response development in 2011-2012, but now represents a limited part 
of STOR (less than 10%) and 9-10 aggregators have left, after the requirements have 
become more and more challenging for consumers (daily weekday participation required 
with a window of 11-13 hours per day, in order to be paid at a competitive level). Two 
new variations, STOR Premium Flexible and STOR Runway (auction of 78MW in 2017, 
which represents 0.1% of GB total electricity generation capacity), have been designed 
to provide better opportunities for Demand Response aggregation in STOR. In 2012- 
2013, 3178.3 MW were contracted under STOR and 167.2 GWh were used. In 2013-2014, 
3097 MW were contracted under STOR and 292.5GWh were used. In 2014-2015, 3500 
MW were contracted under STOR and 233GWh were used. The data do not disentangle 
the share of demand response and generation. The TSO provided the following example 
where the use of STOR successfully contributed to preventing a significant supply 
disruption. In this particular event (see Table 10), the total amount of demand reduction 
expected under STOR and instructed by the TSO to the DNOS was of 1325 MW, and the 
total demand reduction achieved is estimated at 765MW equating to approximately 60% 
of their expected demand reduction. 


Faunhofer ISE Energy Charts, Net installed electricity generation capacity in Germany. Available at: 
https://www.energy-charts.de/power_inst.htm 

5? UK Government, 2017 Digest of UK Energy Statistics, Electricity. Available at: 
https://www.gov.uk/government/uploads/system/uploads/attachment data/file/633779/Chapter 5.pdf 
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Table 10 : Example of prevention of supply disruption with STOR 


Event 1. On Saturday 11th February 2012, a number of generation losses occurred during the morning period combined with effects from the 
DATI) under-estimated demand due to the drop in temperatures across GB, resulting in the National Grid Control Room issuing system 
warnings to the industry and instructing Demand Control to Distribution Network Operators (DNOs). 

2. At 10:10 demand control instructions were issued to reduce demand; Demand control imminent issued to all DNOs and Risk of system 
disturbance issued to industry participants. 

3. At 12:48 inadequate system margin (NISM) was issued for period 17:30-19:30. The warning of Risk of system disturbance was also 
extended to cover the darkness peak period. 

4. At 18:30 both inadequate system margin and risk of system disturbance were cancelled. 


STOR Time MW Commentary 
PVE Eis 08:30 282MW Total of 282MW of STOR was instructed 
/ 08:36 407MW Total of 407MW of STOR was instructed 
Utilisation [y:7y 895MW Total of 895MW of STOR was instructed 
during the [RT 1207MW Total of 1207MW of STOR was instructed 
event 09:30 1243MW Total of 1243MW of STOR was instructed 
09:47 1423MW Total of 1423MW of STOR was instructed 
09:47 1610MW Total of 1610MW of STOR was instructed 
10:05 1760MW Total of 1760MW of STOR was instructed 
10:06 National Grid Control Room began issuing Demand Control Instructions to DNOs 
10:08 1910MW Total of 1910MW of STOR was instructed 
10:25 A Demand Control Imminent warning was issued by National Grid Control Room; A Risk of System Disturbance 
warning was also issued 
10:26 1588MW Total of 1588MW of STOR was instructed 
11:00 1373MW Total of 1373MW of STOR was instructed 
11:15 National Grid Control Room started to issue instructions to cancel all Demand Control 
12:00 1271MW Total of 1271MW of STOR was instructed 
12:15 757MW Total of 757MW of STOR was instructed 
12:30 357MW Total of 357MW of STOR was instructed 
12:45 All STOR ceased, The Demand Control Imminent warning was cancelled 


Source: National Grid 


Note: The amount of STOR at a certain point of time in the above table is the total amount of STOR that were instructed prior to and / or at that particular moment and valid 
for that particular time. 
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The Demand-Side Balancing Reserve (DSBR), introduced in the 2015-2016 winter, is 
targeted at large energy users who volunteer to reduce their demand during winter 
weekday evenings between 4 and 8 pm in return for a payment. 319MW of DSBR were 
contracted in 2014-2015, and 515MW in 2015-2016. In 2016, DSBR represented 0.7% of 
GB total electricity generation capacity. There was no DSBR procurement in 2016-2017 
and the service will not be renewed as the recent capacity mechanism includes a similar 
availability-requirement rendering this programme obsolete. The TSO gave the following 
example of DSBR activation which successfully contributed to the prevention of a 
significant supply disruption. 


Table 11 : Example of prevention of supply disruption with DSBR 


1. On 4th November 2015 at 13:30, a Notification of Insufficient System Margin (NISM) was 
De ptio issued detailing a shortfall of 500MW from the required system margin over darkness peak 
of the day (running from 16:30 to 18:30). 
2. Margins were expected tight throughout Monday and Tuesday with the extended low 
wind. A large amount of generation was on breakdown through to Mon and Tue, plus a few 
generations delayed return with new breakdowns accounted for additional unavailabilities. 
3. The market warning was cancelled at 17:45. 


OR Time MW Commentary 
—— BR 
; s 13:30 NISM was issued, stating a 500MW shortfall against margin requirements 
15:16 43MW Notification to dispatch Demand Side Balancing Reserve (DSBR) 


was issued over the peak between 17:00 to 18:00 


16:54 134MW National Grid Control Room confirmed that Supplemental Balancing 
Reserve (SBR) were made available (134M for the Darkness Peak 
(DP around 17:30). 


17:2 400MW STOR ran over the DP, whilst SBR not instructed. 
17:20 43MW DSBR ran over the DP. 
17:45 NISM was cancelled 


Source: National Grid 


The Frequency Control by Demand Management (FCDM) programme consists in 
bilateral contracts between demand side providers and the TSO to manage large 
deviations in frequency, such as those caused by the sudden loss of a large generating 
unit, by interrupting demand consumers. There were nine events in 2013 and nine in 
2014, always with a maximum duration of 30 minutes, leading to the use of the FCDM 
programme. The volume contracted and frequency of activation of the FCDM were not 
publicly available. 


e Greece 
Two Demand Side interruptible programs have been launched in 2016, each of them 
capped to a volume of 1000MW. The results of the 7 auctions so far are as follows: 
500MW in March 2016, 650MW in April 2016, 750MW in May-September 2016, 550MW in 
October-December 2016, 750MW in January-March 2017, 500MW in April-June 2017, 
580MW for July-September 2017. In average, the volume contracted in 2016 represented 
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3.7% of the total installed capacity in Greece.?^ Due to the natural gas crisis that affected 
Greece in December 2006-January 2017, power reduction orders were issued on 
23.12.2016 and 10.01.17 respectively, but the amount of supply margin created is not 
available. 


e Ireland 

The interruptible loads programme STAR (Short-Term Active Response) provides the 
TSO with 45MW of static reserve from industrial sites used in the event of system 
frequency falling below 49,3 Hz. In 2014, the volume contracted represented 0.6% of the 
total electricity capacity in Ireland.?? 21 events led to the activation of STAR since 2010, 
offering a supply margin between 17 MW and 89 MW (average of 43 MW). The TSO 
considers the STAR scheme effective to prevent or reduce electricity supply disruptions, 
however the programme will be replaced in 2018 by a new programme allowing 
consumers to participate in ancillary services. Some information on the STAR events can 
be found below, however the level of supply margin was only available for recent years. 


Table 12 : Activations of the STAR programme 


31/05/2016 14:29 
20/05/2016 21:25 
09/04/2016 23:28 
07/07/2015 
22/06/2015 
14/03/2015 16:51 
27/04/2014 09:40 
22/04/2014 17:30 
10/05/2013 11:40 
06/05/2013 04:10 
06/11/2012 19:30 
03/09/2012 04:38 
20/08/2012 14:50 
29/11/2011 11:28 
09/10/2011 12:25 
05/09/2011 11:39 
25/05/2011 15:50 
27/11/2010 10:15 
04/07/2010 12:27 
01/05/2010 09:08 
27/04/2010 15:39 
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Source: EirGrid 


e Italy 


The interruptible contracts programme is a dedicated Demand Response programme 
with a current enrolment of about 4 GW. For the 2015-2017 period, the capacity 
contracted was 3300 MW in mainland Italy, 389 MW in Sicilia, 372 MW in Sardinia. In 
2015, the volume contracted represented 3.4% of the total electricity generation capacity 
in Italy.°° It is unclear whether the programme has ever been activated. 


e Netherlands 


Consumer loads can participate since 2014 in the emergency capacity 
("Noodvermogen"), contracted annually. In 2015, 350 MW were procured for the 
emergency capacity and 150 MW for its variation called Omgekeerd Noodvermogen. In 
2016, 350 MW were procured in total. In 2016, the volume contracted represented 1.2% 
of the total electricity installed capacity in the Netherlands.°” The emergency capacity has 


Energypedia, Greece Energy Situation. Available at: 
https://energypedia.info/wiki/Greece Energy. SituationzzEnergy. Supply 

Eirgrid, Generation Capacity Statement 2016-2025. Available at: http://www.eirgridgroup.com/site- 
files/library/EirGrid/Generation Capacity Statement 20162025 FINAL.pdf 

A2A, Italian Energy Market Overview 2015-2016. Available at: https://s3-eu-west-1.amazonaws.com/a2a- 
be/a2a/2017-03/Overview-Italian-Energy-Market-2015-2016.pdf 

TenneT, Record low electricity prices in first 8 months of 2016. Available at: 
https://www.tennet.eu/news/detail/record-low-electricity-prices-in-first-8-months-of-2016/ 
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been used 19 times in 2013 and 27 times in 2014. The supply margin thus created was 
not available. 


e Poland 

Demand Response can participate in the Emergency Demand Response Programme 
(EDRP). The first contract for EDRP was signed in March 2013 (30 MW of capacity for 
summer and 25 MW for winter). In total 6 auctions were organised between 2012-2015 
and the cumulative power contracted exceeded 200 MW (0.8% of peak demand and 0.5% 
of total installed capacity*’) in 2016. However, having only utilisation payments proved to 
be unattractive to customers and by the seventh tender in mid-end-of 2016 there were 
no participants. The EDRP was activated 3 times over the 2010-2016 period, all between 
July and September 2015 during a demand peak, for a supply margin of 1.5 GWh in total. 
The TSO considers this service effective to contribute to security of supply and is planning 
to organise more tenders, with a maximum of offers without availability payment, and 
about 500 MW with both availability and utilisation payments. 


e Portugal 
The interruptible contracts programme is limited to large industrial consumers 
(contracted power above 4 MW), and represents an available capacity of 2000 MW of 
demand reduction in peak hours. In 2016, the volume available represents 10.2% of the 
total installed electricity capacity in Portugal.°? The TSO Portugal has never activated 
these contracts in the last 15 years. 


e Spain 

The interruptible load programme for large industrial customers (contracted power 
above 5 MW) acts as an emergency action in case the system is lacking generation and 
the balance resources are not enough. It represents an available capacity of 2000 MW of 
demand reduction in peak hours. The interruptible load capacity was of 3020 MW in 2015 
and 2890 MW in 2016. In 2016, the volume contracted represented 2.7% of the total 
installed electricity capacity in Spain. The lack of activation of the Interruptible Load 
programme in the last decade raises questions about being a form of subsidy to the 
national industry. In 2016, some interruptible demand was curtailed due to local 
problems in the transmission grid, but the margin supply was not available. 


4.4 Output 6 - The value of lost load to society from significant 


disruption events 


Output 6 provides information on the value of lost load (VoLL) resulting from the 
significant electricity supply disruption events. Modern industrialized societies are heavily 
dependent on electricity. That holds both for commercial and private users of electricity. 
Electricity is an essential input factor to almost all economic processes, meaning that 


Index Mundi, Poland Electricity = installed generating capacity. Available at: 

http://www.indexmundi.com/poland/electricity_installed_generating_capacity.html 

*? Global Legal Insights, Portugal Energy 2018. Available at: https://www.globallegalinsights.com/practice- 
areas/energy-laws-and-regulations/portugal 

60 REE, Red Eléctrica publishes the ‘Spanish Electricity System Report 2016. Available at: 

http://www.ree.es/en/press-office/press-release/2017/06/red-electrica-publishes-spanish-electricity-system- 

report-2016 


most commercial activities will come to a stand-still in case of a disruption. Also, many 
leisure activities are based on electricity and cannot be carried out during a disruption. 


Although being expressed in monetary terms, the VoLL cannot be derived from market 
interaction, as there is no market for trading disruptions. The VoLL must therefore be 
found using scientific measuring techniques such as: 


e Stated preferences methods: obtain the information directly from the consumers. 
Surveys can be used to ask for the willingness to pay (WTP) for avoiding a 
disruption, or the willingness to accept (WTA) for experiencing a disruption. The 
surveys can be hypothetical (ex ante) or refer to an actual blackout or brownout in 
the past (ex post). 


e Revealed preferences methods: use data to quantify the consumers loss. Such 
quantifications can for example be based on the production function of an 
industry, or the household income as a proxy for the value of leisure for private 
consumers. Undertaken mitigation measures can also give an indication of how 
costly it is to the consumer to be out of electricity. 


4.4.1 Objective and approach 


The objective of this chapter is to get a better understanding of the welfare loss, or in 
other words the socio-economic impact of the disruptions that occur in Europe. 


The methodology applied to obtain a value for the value of lost load (VoLL) per non- 
supplied kWh of electricity for each of the 28 Member States involved 4 main steps. First, 
we carried out a literature review to obtain a range of VoLL estimates across countries 
and across the two categories households and non-households. We identified and 
removed the outlier estimates and found a mean for the remaining estimates. Then, we 
extrapolated the mean to the 28 Member States. Finally, we merged the household and 
non-household VoLL into one estimate per country. 


4.4,2 Findings 


Commercial and private consumers face different types of costs due to disruptions. Being 
out of power causes direct and indirect damage costs, and trying to avoid the damage 
costs causes mitigation costs, see Table 13. The VoLL reflects those costs. 


Damage costs and mitigation costs are often exclusive, meaning a consumer faces either 
the one or the other type. If the consumer spent money on a standby generator or 
battery, then he or she will not face damage costs. Mixes are however also possible, for 
example if a disruption is longer than the maximum running time of the generator or 
battery. 


Damage costs are typically higher for commercial consumers than for private individuals, 
as one can argue that when there is no power, all commercial activity ceases, meaning no 
production or other forms of value creation is possible. In addition, products, machines 
and devices can be damaged, and data can be lost. For private individuals, a disruption 
affects the leisure time, which will be very restricted when there is no power, but will not 
cease fully. Also damage to goods (e.g. food) and data loss can occur in private 
households, but the damaged goods are typically of a lower value. 
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Worth noting is that the cause of the disruption does not have any impact on those costs. 
That does not imply that knowledge about what drives disruptions is redundant, on the 
contrary, understanding the causes will help design policies to minimise the occurrence of 
disruptions. 


Table 13 : Damage and mitigation costs related to disruptions 


Damage costs Mitigation costs 


Indirect 


(a) opportunity costs of idle 
resources: labour, 
country, capital, profits 

dote ai (bD) production hold-ups and 

consumers, restart times 


(a) delayed deliveries along 
the value chain 
(b) damage for consumers if Procurement of standby 


the company produces an generators, batteries etc., 
investment in grid 
end product 


industry (c) adverse effects and (c) ts/b fits f construction via charges 
C) costs/benefits for some ; 
damage to capital goods, bu (network tariffs) 
data loss 


(d) health and safety aspects (d) health and safety aspects 


(a) Restrictions on activities, 


IBSEUISU ep stress Procurement of standby 


private (b) financial costs due to Restrictions on acquisition of | generators, batteries etc., 

consumers, damage to premises and | goods, costs for other private | Investment in grid 

households real estate, food individuals and companies construction via charges 
spoilage, or data loss (network tariffs) 

(c) health and safety aspects 


Source: Schróder and Kuckshinrichs (2015) Value of Lost Load: An Efficient Economic Indicator for Power Supply Security? A 
Literature Review. 


Existing studies prove that estimating the VoLL is a difficult exercise. The reported VoLL 
estimates vary significantly. The estimates for households range from 2 EUR per non- 
supplied kWh in a study for Austria to 72 EUR per kWh in a study for Ireland, which is 
more than 30 times larger than the Austrian estimate. For non-households, the range is 
even larger. Here, the smallest estimate is 0.2 EUR for the gas and water sector in 
Cyprus, the largest 216 EUR per kWh for non-households in Austria, see Table 14. 


The USA has been added to the table, as comparing the EU estimates to US-American 
estimates will be interesting for two reasons: firstly, several studies have been conducted 
for the US. Secondly, the USA is a modern, industrialised country and in that regard 
similar to the EU Member States. The additional estimates for the US will help understand 
which of the European estimates (if any) should be considered outliers. 
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Table 14 : Existing estimates for the Value of Lost Load (VoLL), 2010 prices 


country 


Austria 


Germany 


Ireland 


Netherlands 


Spain 


Sweden 


United 
Kingdom 


USA 


Households 
Source, details, 


Non-households 


EUR/kWh methodology EUR/kWh Source, details, methodology 
2.24 Fickert (2004) 
Bliem (2007), Bliem (2007), service, 
aon Macroeconomic approach 216 10 Macroeconomic approach 
3.46 Reichl et al. (2007) 7.80 Reichl et al. (2007), service 
2.45 Reichl et al. (2013), winter, 26.80 Reichl et al. (2013), winter, 
t Macroeconomic approach t Macroeconomic approach 
Zachariadis and Rey (2012), 
TA lower boundary: Gas and water 
Zachariadis and Rey (2012), , : 
9.22 Macroeconomic approach 0.20-119.96 supply, upper boundary: 
Construction, Macroeconomic 
approach 
«zs Praktiknjo et al. (2011), 
Praktiknjo et al. (2011), à : ; 
: industrial, Macroeconomic 
18.41 Macroeconomic approach 7.04 approach combined with Monte 
combined with Monte Carlo Carlo 
Growitch et al (2013), Growitch et al (2013), 
13.27 | 8.32 ; 
Macroeconomic approach Macroeconomic approach 
Rópke, L. (2013), industry, 
z lower boundary: Trade and 
15.34 die a road 2.86-15.67 services, upper boundary: 
PP transport, Macroeconomic 
approach 
71.63 bon ed Macroeconomic 
Tol (2007), M i 
Cer and Niaur (2009), 8.43 - 75.85 es Jy Macroeconomic 
10.27 calculation based on 
t estimated peak price of 
planned electricity capacity 
4.1 Bertrazzi et al. (2005), 129.91 Bertrazzi et al. (2005), service 
survey 
De Nooij et al. (2007), 
19.13 | " 
Macroeconomic approach 6.94 De Nooij et al. (2007), 
3.86 Baarsma and Hop (2009), E Macroeconomic approach 
t Survey 
22.77 Wilks and Bloemhof (2005), 
Survey 
Linares et al (2012), lower 
8.27 Linares et al (2012), 0.92-34.02 boundary: metal, upper 
Macroeconomic approach boundary: Construction, 
Macroeconomic approach 
Anderson and Taylor 
3-76 (1985), Survey 
1.64 London Economics, 2013, 
manufacturing 
Doane et al. (1988), winter, Fisher (1986), summer, 
zh evening 20:78 afternoon, trade 
Doane et al. (1988), Woo & Gray (1987), summer, 
Tu summer, afternoon 70 afternoon, production 
Sanghvi (1983), summer, Woo & Train (1988), summer, 
0.21 midday 10.20 afternoon, trade 
3.86 Balducci, Roop et al. (2002), 26.86 Carves et al. (1990), service 


Survey 


(max value shown) 
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Households Non-households 


Source, details, 
country EUR/kWh methodology EUR/kWh Source, details, methodology 


Burns and Gross (1990), Doane et al., (1990), winter, 


6.03 8.03 


Survey evening, production 
2.59 Krohm (1978), Survey 45.94 Sullivan, (1996), service 
gau lawton, Sullivan et al 7.62 Sullivan (1996), production 


(2003), Survey 


Note: all estimates have been converted into 2010 prices using the price index of the Euro area 2002-2012. 


Source: The studies listed in the table. 


The large variation in estimates can at least partly be explained through: 


e Different methodological approaches. Having different strengths and 
weaknesses, different approaches (e.g. stated vs. revealed preferences, actual vs. 
hypothetical disruption, willingness to pay vs. willingness to accept etc) typically 
leads to different results. 


e Different characteristics of the disruptions assessed. The magnitude of the 
damage costs depends significantly on the disruption’s time of day, season and 
duration, as well as the fact of whether the disruption is planned or forced. If a 
disruption is planned and announced sufficiently in advance, then companies can 
make arrangements to avoid high damage costs, and private consumers can plan 
their leisure activities accordingly. 


e Differences between countries. Different income and price levels as well as 
differences in the industry structure between countries will entail different levels of 
damage costs. 


e Different preferences of private consumers. Also within one country, and 
private individuals’ damage costs will vary for two main reasons: firstly, leisure 
time is scarcer for some than for other individuals. The scarcer, the higher the 
damage costs of restricted leisure activities. Secondly, personal preferences 
regarding the use of their leisure time will vary, also given the same level of 
scarcity. The damage costs will for example be higher for an individual who spends 
his/her free time streaming movies, than for an individual going for walks in the 
forest. 


e Different production functions for industrial consumers. Different 
commercial users are, depending on their production function, to a different level 
dependent on electricity as an input. Also, the extent to which machinery, devices 
and products can be damaged as a result of a disruption depends on the particular 
industry or business model. 


Given this extreme variance in existing estimates, it is impossible to declare a single 
number as the correct VoLL for Europe. However, looking at the variation in estimates 
can give an indication of the most likely range of a VoLL for the EU. 


15 out of the 16 estimates for the VoLL for households in European Member States lie 
comparably close to each other, between 2 and 23 EUR per kWh. This range is similar to 
the US-American estimates, which range between 0.2 and 20 EUR per kWh, see Figure 
52. The 16" estimate however is 72 EUR per kWh (marked in red in the figure) and 


113 


seems to be a clear outlier. Excluding the outlier results in an average household VoLL of 
9 EUR per kWh, and a median of 8 EUR per kWh. 


The estimates for non-households show a larger variance, probably partly due to the 
large differences in different industries’ dependency on electricity. Also, the estimates for 
the USA vary more for commercial than for private consumers. The estimates for 
European Member States range from 0.2 EUR per kWh to 216 EUR per kWh, the latter 
value clearly seeming to be an outlier (marked in red in the figure). Removing that outlier 
results in a range of 0.2 to 130 EUR per kWh, with an average of 30 EUR per kWh anda 
median of 8 EUR per kWh. This range seems reasonable when comparing to the USA. The 
lowest value of 0.2 EUR per kWh seems implausibly low, but is a lower boundary of a 
study for Cyprus, and enters the calculations together with the study’s upper boundary of 
120 EUR per kWh. The average and median values are therefore not downward-biased 
from that study. 


Figure 52 : Variation in the estimates for the Value of Lost Load (VoLL), 2010 prices 


250 EUR/kWh 


200 
150 range: range: __ range: range: 
2.34 - 22.77 0.21 - 20.03 — 0,20 - 129.91 7.62 - 71.63 

100 average: average: average: average: 

— 9.5 8.7 = 29.8 = 27.3 

50 median: median: — median: — median: 
| | 8.3 — 6.0 = 8.3 = 20.8 
0 m = 
EU USA EU USA 
Households Non-households 


Note: all estimates in 2010 prices. The two outliers marked in red are not included in the range, average and median. 


Source: Copenhagen Economics based on Table 14 above. 


To get an idea of how much socio-economic harm disruptions do to different countries, 
the overall estimates can be adjusted for the 28 Member States of the EU. 


For households, the GDP per capita is a suitable distribution key for the VoLL. For non- 
households, we suggest a distribution key based on the value added per consumed unit of 
electricity, see Table 16. 


This indicator itself can be interpreted as a very simplistic macroeconomic estimate for 
the VoLL for non-households, as it reflects how much value creation will not be possible 
per non-supplied kWh (assuming that all economic activity ceases without power). The 
range of this simplistic non-households VoLL is from 2 EUR per kWh for Bulgaria to 14 
EUR per kWh for Ireland (2015 prices). This range is in line with the lower part of the 
variance in Table 15. It does, however, only capture the average costs due to non- 
production, not the costs due to damaged products or machinery; those estimates should 
therefore rather be interpreted as the lower boundary. 
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As the starting point to be adjusted for country differences, one could either use the 
average or the median of the VoLL estimates. For private consumers, average and 
median are very close to each other (9 and 8 EUR/kWh). For commercial consumers, the 
average is significantly higher than the median (30 and 8 EUR/kWh). We will use the 
average VoLL for two reasons: firstly, a comparison with the US estimates, which have an 
average of 21 EUR/kWh and a median of 27 EUR/kWh, suggests that the EU-median of 
non-household VoLL might be too low. Secondly, the VoLL for industrial and commercial 
consumers tends to be considerably higher than for private users according to the 
literature (e.g. Schróder 2015), suggesting that the average is more reasonable. 


The two VoLL averages are based on studies for 9 different Member States (Austria, 
Cyprus, Germany, Ireland, Italy, the Netherlands, Spain, Sweden and the UK). To make 
the averages suitable for taking them as a starting point on EU-level, they are adjusted to 
2016 prices and scaled based on the average "key" of the 9 countries, see Table 15. 


Table 15 : Scaling the average VoLL estimates toward an EU28 estimate 


ousehold 9 10 129 8 


n 30 32 123 26 


Note: The conversion from 2010 to 2016 prices has been made based on the average inflation rate of the 9 countries with 
studies. The average index of countries with studies shows a simple average of the indexes of the 9 countries with studies. The 
table shows rounded values. 


Source: Copenhagen Economics calculation based on Eurostat inflation data. 


The scaling results in an EU-level VoLL estimate for households of 8 EUR per non-supplied 
kWh of electricity, and 26 EUR per kWh for non-households. Those EU28 averages are 
then adjusted to the 28 Member States using the distribution keys explained above. This 
adjustment results in a distinct VoLL estimate for households and non-households for 
each of the Member States, see Table 16. 


The Member States' household-VoLL ranges from 2 EUR/kWh (Bulgaria) to 32 EUR/kWh 


(Luxembourg). The VoLL for commercial consumers ranges from 11 EUR/kWh (Bulgaria) 
to 67 EUR/kWh (Ireland). 
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Table 16 : Distribution keys for VoLL estimates across countries 


Households Non-households 


GDP per Average VoLL Value added 
capita distribution scaled to [EUR] per distribution 
(current key country consumed key 
prices, (EU28=100) (rounded, kWh of (EU28=100) 
2016) EUR/kWh) electricity 


Average VoLL 
scaled to 

country 
(EUR/kWh) 


EU-28 
Austria 
Belgium 
Bulgaria 
Croatia 


Cyprus 


Czech 
Republic 


Denmark 
Estonia 
Finland 

France 

Germany 

Greece 

Hungary 
Ireland 
Italy 

Latvia 
Lithuania 
Luxembourg 
Malta 
Netherlands 
Poland 
Portugal 
Romania 
Slovakia 
Slovenia 
Spain 
Sweden 


United 
Kingdom 


Note: VoLL in 2016 prices. The value added per consumed kWh of electricity for non-households is a weighted average of that 
indicator for industries and other sectors. The value added per kWh is in 2015 prices. 


Source: Copenhagen Economics based on Eurostat data. 
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An average VoLL per country can be derived from weighting household- and non- 
household VoLL according to the electricity consumption of those two categories in the 
respective country, see Table 17. 


Table 17 : An average VoLL for each of the EU Member States 


Country-VoLL 

(weighted 

average, 

rounded, 2016 
prices) 


Average VoLL Share at total Average VoLL Share at total 
scaled to electricity scaled to electricity 
country consumption in country consumption in 
(EUR/kWh) the country (EUR/kWH) the country 


Source: Copenhagen Economics based on Eurostat data. 
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Combining the average EU VoLL estimate and the rough estimates for electricity non- 
supplied from output 4 (Figure 42), we can get an overall understanding of how costly 
disruptions have been to society. Important to note, this analysis combines two very 
uncertain estimates based on a range of assumptions, and must therefore be interpreted 
accordingly. The socio-economic cost from disruptions in Europe seems to be around 10 
to 25 billion EUR every year, see Figure 53.9! The true costs to society is the VoLL minus 
the electricity prices, as consumers do not pay for the electricity they have not been 
supplied with. 


Figure 53 : Rough estimate of the total VoLL in Europe due to disruptions, 2010-2014 


billion EUR 
25 
20 
20 
15 


10 


2010 2011 2012 2013 2014 


Source: Disruptions data based on CEER, VoLL calculations by Copenhagen Economics, based on literature and Eurostat data. 


4.4.3 Conclusions 


Disruptions are costly to society, as for every disruption, consumers willing to pay the 
price for electricity do not have the opportunity to consume it. That limits the production 
of commercial consumers and the use of leisure of private consumers, and can in addition 
lead to damages of products or machinery. The value of lost load (VoLL) is an estimate 
for those costs. 


The VoLL depends on a wide range of factors. It varies for example depending on the 
time of day, season and duration of the disruption, as well as with the individual 
preferences and production functions of the private and commercial consumers affected. 
The VoLL is moreover typically higher for countries with a high-income level, higher for 
commercial than for private consumers, and higher for forced than for planned 
disruptions. 


In addition, different methodological methods to measure the VoLL will lead to different 
results. It is for those reasons very difficult to estimate the VoLL, and existing estimates 
vary immensely. All estimates, and EU averages in particular, should therefore be 
considered an approximation rather than the "true" cost to society. 


Based on an assessment of existing estimates combined with a  well-founded 
extrapolation, we find that the household-VoLL is around 5-10 EUR/kWh at the European 


$! Not taking into account potential cost savings from not generating the electricity 
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average, meaning that a one kWh of non-supplied electricity to an average household in 
Europe implies costs to that consumer of 5 to 10 EUR. For non-households like industrial 
or commercial consumers, we find an average VoLL of 20-30 EUR/kWh, which is 
approximately three times as much as for households. 


Those estimates show that the disruptions in Europe in the period 2010 to 2014 gave rise 
to a socio-economic loss of approximately 10 to 25 billion EUR annually. 
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5. LIST OF ANNEXES 


5.1 Annex 1 - Content of Commission Regulation (EU) 543/2013 


Table 18 Content of Regulation 543/2013 


Article Regulation text Deadline 

6.1.A the total load per market time unit one hour after the operating 

period 

6.1.B a day-ahead forecast of the total load per market time unit two hours before the gate 

closure 

6.1.C a week-ahead forecast of the total load for every day of the each Friday no later than two 
following week, which shall for each day include a maximum and | hours before the gate closure 
a minimum load value of the day-ahead market in the 

bidding zone 

6.1.D a month-ahead forecast of the total load for every week of the one week before the delivery 
following month, which shall include, for a given week, a month 
maximum and a minimum load value 

6.1.E a year-ahead forecast of the total load for every week of the the 15th calendar day of the 
following year, which shall for a given week include a maximum month before the year to 
and a minimum load value which the data relates 

7.1.A the planned unavailability of 100 MW or more of a consumption one hour after the decision 
unit, including changes of 100 MW or more in the planned regarding the planned 
unavailability of consumption units, lasting at least one market unavailability is made 
time unit, specifying: 

— bidding zone, 

— available capacity per market time unit during the event, 

— reason for the unavailability, 

— the estimated start and end date (day, hour) of the change in 
availability 

7.1.B changes in actual availability of a consumption unit with a power | one hour after the change in 
rating of 100 MW or more, specifying: actual availability 
— bidding zone, 

— available capacity per market time unit during the event, 

— reason for the unavailability, 

— the start date and the estimated end date (day, hour) of the 
change in availability 

8.1 the year-ahead forecast margin evaluated at the local market one week before the yearly 
time unit capacity allocation but no later 

than the 15th calendar day of 
the month before the year to 
which the data relates. 

9.1 information on future changes to network elements and one week before the yearly 
interconnector projects including expansion or dismantling in capacity allocation but no later 
their transmission grids within the next three years, to the than the 15th calendar day of 
ENTSO for Electricity. This information shall only be given for the month before the year to 
measures expected to have an impact of at least 100 MW on which the allocation relates. 
cross zonal capacity between bidding zones or on profiles at The information shall be 
least during one market time unit. The information shall include: | updated with relevant changes 
(a) the identification of the assets concerned; before the end of March, the 
(b) the location; end of June and the end of 
(c) type of asset; September of the year to 
(d) the impact on interconnection capacity per direction between | which the allocation relates. 
the bidding zones; 

(e) the estimated date of completion 
10.1.A the planned unavailability, including changes in the planned one hour after the decision 


unavailability of interconnections and in the transmission grid 
that reduce cross zonal capacities between bidding zones by 100 
MW or more during at least one market time unit, specifying: 

— the identification of the assets concerned, 

— the location, 


regarding the planned 
unavailability is made 
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Article Regulation text Deadline 
— the type of asset, 
— the estimated impact on cross zonal capacity per direction 
between bidding zones, 
— reasons for the unavailability, 
— the estimated start and end date (day, hour) of the change in 
availability 
10.1.B changes in the actual availability of interconnections and in the one hour after the change in 
transmission grid that reduce cross zonal capacities between actual availability 
bidding zones by 100 MW or more during at least one market 
time unit, specifying 
— the identification of the assets concerned, 
— the location, 
— the type of asset, 
— the estimated impact on cross zonal capacity per direction 
between bidding zones, 
— reasons for the unavailability, 
— the start and estimated end date (day, hour) of the change in 
availability 
10.1.C changes in the actual availability of off-shore grid infrastructure one hour after the change in 
that reduce wind power feed-in by 100 MW or more during at actual availability 
least one market time unit, specifying 
— the identification of the assets concerned, 
— the location, 
— the type of asset, 
— the installed wind power generation capacity (MW) connected 
to the asset, 
— wind power fed in (MW) at the time of the change in the 
availability, 
— reasons for the unavailability, 
— the start and estimated end date (day, hour) of the change in 
availability 
11.1.A the forecasted day-ahead transfer capacities per direction - 
between bidding zones in case of coordinated net transmission 
capacity-based capacity allocation 
11.1.A the forecasted week-ahead transfer capacities per direction each Friday, for all days of the 
between bidding zones in case of coordinated net transmission following week 
capacity-based capacity allocation 
11.1.A the forecasted month-ahead transfer capacities per direction two working days before the 
between bidding zones in case of coordinated net transmission monthly allocation process for 
capacity-based capacity allocation all days of the following month 
11.1.A the forecasted year-ahead transfer capacities per direction one week before the yearly 
between bidding zones in case of coordinated net transmission allocation process but no later 
capacity-based capacity allocation than 15 December, for all 
months of the following year 
11.1.A the offered intraday transfer capacity per direction between one hour before the first intra- 
bidding zones in case of coordinated net transmission capacity- day allocation and then real- 
based capacity allocation time, for each market time unit 
11.1.A the offered day-ahead transfer capacity per direction between one hour before spot market 
bidding zones in case of coordinated net transmission capacity- gate closure, for each market 
based capacity allocation time unit 
11.1.A the offered week-ahead transfer capacity per direction between one day before the weekly 
bidding zones in case of coordinated net transmission capacity- allocation process 
based capacity allocation 
11.1.A the offered month-ahead transfer capacity per direction between | two working days before the 
bidding zones in case of coordinated net transmission capacity- monthly allocation process 
based capacity allocation 
11.1.A the offered year-ahead transfer capacity per direction between one week before the yearly 
bidding zones in case of coordinated net transmission capacity- allocation process but no later 
based capacity allocation than 15 December 
11.1.A the offered other transfer capacity per direction between bidding | - 


zones in case of coordinated net transmission capacity-based 
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Article 


Regulation text 


capacity allocation 


Deadline 


11.1.A the offered day-ahead transfer capacity implicit per direction one hour before spot market 
between bidding zones in case of coordinated net transmission gate closure, for each market 
capacity-based capacity allocation time unit 

11.1.A the offered intraday transfer capacity implicit per direction one hour before the first intra- 
between bidding zones in case of coordinated net transmission day allocation and then real- 
capacity-based capacity allocation time, for each market time unit 

11.1.B the relevant flow-based parameters in case of flow based sufficiently in advance of the 
capacity allocation allocation process 

11.3 In relation to direct current links, TSOs shall provide updated one hour after the information 
information on any restrictions placed on the use of available is known to the ENTSO for 
cross-border capacity including through the application of Electricity 
ramping restrictions or intraday transfer limits 

11.4 TSOs or, if applicable, transmission capacity allocators, shall - 
provide a yearly report to the ENTSO for Electricity indicating: 

(a) the main critical network elements limiting the offered 
capacity; 

(b) the control area(s) which the critical network elements 
belong to; 

(c) the extent to which relieving the critical network elements 
would increase the offered capacity; 

(d) all possible measures that could be implemented to increase 
the offered capacity, together with their estimated costs. 

When preparing the report TSOs may choose not to identify the 
asset concerned and specify its location if it is classified as 
sensitive critical infrastructure protection related information in 
their Member States as provided for in point (d) of Article 2 of 
Directive 2008/114/EC 

12.1.A in case of explicit allocations, for every market time unit and one hour after each capacity 
per direction between bidding zones: allocation 
— the capacity (MW) requested by the market 
— capacity (MW) allocated to the market, 

— the price of the capacity (Currency/MW), 
— the auction revenue (in Currency) per border between bidding 
zones 

12.1.B for every market time unit and per direction between bidding two hours after each round of 
zones the total capacity nominated nomination 

12.1.C prior to each capacity allocation the total capacity already at the latest when publication 
allocated through previous allocation procedures per market of offered capacity figures 
time unit and per direction become due as set out in the 

Annex 

12.1.D for every market time unit the day-ahead prices in each bidding one hour after gate closure 
zone (Currency/MWh) 

12.1.E in case of implicit allocations, for every market time unit the net | one hour after each capacity 
positions of each bidding zone (MW) and the congestion income allocation 
(in Currency) per border between bidding zones 

12.1.F scheduled day-ahead commercial exchanges in aggregated form | one hour after the last cut-off 
between bidding zones per direction and market time unit time and, if applicable, shall be 

updated no later than two 
hours after each intra-day 
nomination process 

12.1.G physical flows between bidding zones per market time unit one hour for each market time 

unit as closely as possible to 
real time but no later than one 
hour after the operational 
period 

12.1.H cross zonal capacities allocated between bidding zones in one hour after the allocation 
Member States and third countries per direction, per allocated 
product and period. 

13.1.A information relating to redispatching per market time unit, one hour as soon as possible 
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Article Regulation text Deadline 
specifying: but no later than one hour 
— the action taken (that is to say production increase or after the operating period, 
decrease, load increase or decrease), except for the reasons which 
— the identification, location and type of network elements shall be published as soon as 
concerned by the action, possible but not later than one 
— the reason for the action, day after the operating period 
— capacity affected by the action taken (MW) 

13.1.B information relating to countertrading per market time unit, one hour as soon as possible 
specifying: but no later than one hour 
— the action taken (that is to say cross-zonal exchange increase | after the operating period, 
or decrease), except for the reasons which 
— the bidding zones concerned, shall be published as soon as 
— the reason for the action, possible but not later than one 
— change in cross-zonal exchange (MW) day after the operating period 

13.1.C the costs incurred in a given month from actions referred to in one month after the end of the 
points (a) and (b) and from any other remedial action referred month 

14.1.A the sum of generation capacity (MW) installed for all existing one week before the end of the 
production units equalling to or exceeding 1 MW installed year 
generation capacity, per production type 

14.1.B information about production units (existing and planned) with one week before the end of the 
an installed generation capacity equalling to or exceeding 100 year for 3 following years 
MW. The information shall contain: 

— the unit name, 

— the installed generation capacity (MW), 
— the location, 

— the voltage connection level, 

— the bidding zone, 

— the production type 

14.1.C an estimate of the total scheduled generation (MW) per bidding 18:00 Brussels time 
zone, per each market time unit of the following day 

14.1.D a forecast of wind and solar power generation (MW) per bidding 18:00 update at 8:00; The 
zone, per each market time unit of the following day information shall be provided 

for all bidding zones only in 
Member States with more than 
1% feed-in of wind or solar 
power generation per year or 
for bidding zones with more 
than 5% feed-in of wind or 
solar power generation per 
year. 

15.1.A the planned unavailability of 100 MW or more of a generation one hour after the decision 
unit including changes of 100 MW or more in the planned regarding the planned 
unavailability of that generation unit, expected to last for at unavailability is made 
least one market time unit up to three years ahead, specifying: 

— the name of the production unit, 

— the name of the generation unit, 

— location, 

— bidding zone, 

— installed generation capacity (MW), 

— the production type, 

— available capacity during the event, 

— reason for the unavailability, 

— start date and estimated end date (day, hour) of the change 
in availability 

15.1.B changes of 100 MW or more in actual availability of a generation | one hour after the change in 


unit, expected to last for at least one market time unit, 
specifying: 

— the name of the production unit, 

— the name of the generation unit, 

— location, 

— bidding zone, 


actual availability 
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Article Regulation text Deadline 
— installed generation capacity (MW), 
— the production type, 
— available capacity during the event, 
— reason for the unavailability, and 
— start date and estimated end date (day, hour) of the change 
in availability 
15.1.C the planned unavailability of a production unit of 200 MW or one hour after the decision 
more including changes of 100 MW or more in the planned regarding the planned 
unavailability of that production unit, but not published in unavailability is made 
accordance with subparagraph (a), expected to last for at least 
one market time unit up to three years ahead, specifying: 
— the name of the production unit, 
— location, 
— bidding zone, 
— installed generation capacity (MW), 
— the production type, 
— available capacity during the event, 
— reason for the unavailability, 
— start date and estimated end date (day, hour) of the change 
in availability 
15.1.D changes of 100 MW or more in actual availability of a production | one hour after the change in 
unit with an installed generation capacity of 200 MW or more, actual availability 
but not published in accordance with subparagraph (b), 
expected to last for at least one market time unit, specifying: 
— the name of the production unit, 
— location, 
— bidding zone, 
— installed generation capacity (MW), 
— the production type, 
— available capacity during the event, 
— reason for the unavailability, and 
— start date and estimated end date (day, hour) of the change 
in availability 
16.1.A actual generation output (MW) per market time unit and per five days after the operational 
generation unit of 100 MW or more installed generation capacity | period 
16.1.B aggregated generation output per market time unit and per one hour after the operational 
production type period 
16.1.C actual or estimated wind and solar power generation (MW) in one hour 16.1.C.: and be 
each bidding zone per market time unit updated on the basis of 
measured values as soon as 
they become available. The 
information shall be provided 
for all bidding zones only in 
Member States with more than 
1% feed-in of wind or solar 
power generation per year or 
for bidding zones with more 
than 5% feed-in of wind or 
solar power generation per 
year; 
16.1.D aggregated weekly average filling rate of all water reservoir and | the third working day following 


hydro storage plants (MWh) per bidding zone including the 
figure for the same week of the previous year 


the week to which the 
information relates. The 
information shall be provided 
for all bidding zones only in 
Member States with more than 
10% feed-in of this type of 
generation per year or for 
bidding zones with more than 
30% feed-in of this type of 
generation per year. 
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Article Regulation text Deadline 
17.1.A rules on balancing including: - 
— processes for the procurement of different types of balancing 
reserves and of balancing energy, 
— the methodology of remuneration for both the provision of 
reserves and activated energy for balancing, 
— the methodology for calculating imbalance charges, 
— if applicable, a description on how cross-border balancing 
between two or more control areas is carried out and the 
conditions for generators and load to participate 
17.1.B the amount of balancing reserves under contract (MW) by the two hours before the next 
TSO, specifying: procurement process takes 
— the source of reserve (generation or load), place 
— the type of reserve (e.g. Frequency Containment Reserve, 
Frequency Restoration Reserve, Replacement Reserve), 
— the time period for which the reserves are contracted (e.g. 
hour, day, week, month, year, etc.) 
17.1.C prices paid by the TSO per type of procured balancing reserve one hour after the 
and per procurement period (Currency/MW/period) procurement process ends 
17.1.D accepted aggregated offers per balancing time unit, separately one hour after the operating 
for each type of balancing reserve period 
17.1.E the amount of activated balancing energy (MW) per balancing 30 minutes after the operating 
time unit and per type of reserve period. In case the data are 
preliminary, the figures shall 
be updated when the data 
become available 
17.1.F prices paid by the TSO for activated balancing energy per one hour after the operating 
balancing time unit and per type of reserve; price information period 
shall be provided separately for up and down regulation 
17.1.G imbalance prices per balancing time unit as soon as possible 
17.1.H total imbalance volume per balancing time unit 30 minutes after the operating 
period. In case the data are 
preliminary, the figures shall 
be updated when the data 
become available 
17.1.1 monthly financial balance of the control area, specifying: three months after the 
— the expenses incurred to the TSO for procuring reserves and operational month. In case the 
activating balancing energy, settlement is preliminary, the 
— the net income to the TSO after settling the imbalance figures shall be updated after 
accounts with balance responsible parties the final settlement 
17.1.J if applicable, information regarding Cross Control Area Balancing | one hour after the operating 


per balancing time unit, specifying: 

— the volumes of exchanged bids and offers per procurement 
time unit, 

— maximum and minimum prices of exchanged bids and offers 
per procurement time unit, 

— volume of balancing energy activated in the control areas 
concerned 


period 
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5.1 Annex 2 - Overview of data items available on the ENTSO-E TP 


Table 19 Overview of data items available on the ENTSO-E TP 


Category on Aride n > . 
TP website rper Data item (49 in total) 
6.1.A Actual Total Load 
6.1.B Day-ahead Total Load Forecast 
6.1.C Week-ahead Total Load Forecast 
"en 6.1.D Month-ahead Total Load Forecast 
6.1.E Year-ahead Total Load Forecast 
8.1 Year-ahead Forecast Margin 
14.1.A Installed Generation Capacity Aggregated 
14.1.B Installed generation capacity per unit 
14.1.C Day-ahead Aggregated Generation 
inet n 14.1.D Day-ahead Generation Forecasts for Wind and Solar 
16.1.A Actual Generation per Generation Unit 
16.1.B Aggregated Generation per Type 
16.1.C Aggregated Generation per Type 
16.1.D Aggregate Filling Rate of Water Reservoirs and Hydro Storage Plants 
9.1 Expansion And Dismantling Projects 
11.1.A Forecasted/offered Transfer capacities 
11.1.B Day Ahead Flow Based Allocations 
11.3 Cross-border Capacity for DC Links 
11.4 Yearly Report About Critical Network Elements Limiting Offered Capacities 
12.1.A Explicit Allocations - Use of the Transfer Capacity 
Transmission 12.1.B Total Nominated Capacity 
12.1.C Total Capacity Already Allocated 
12.1.D Day-ahead Prices 
12.1.E Implicit Allocations - Net Positions 
12.1.F Scheduled Commercial Exchanges 
12.1.G Physical Flows 
12.1.H Transfer Capacities Allocated with Third Countries 
17.1.A Rules on Balancing 
17.1.B Amount of Balancing Reserves Under Contract 
17.1.C Price of Reserved Balancing Reserves 
17.1.D Accepted Aggregated Offers 
Balancing 
17.1.E Activated Balancing Energy 
17.1.F Prices of Activated Balancing Energy 
17.1.G Imbalance Prices 
17.1.H Total Imbalance Volumes 


Category on 
TP website 


Article in 


Regulation 
543/2013 
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Data item (49 in total) 


17.1.1 Financial Expenses and Income for Balancing 
17.1.J Volumes of Exchanged Bids and Offers 
7.1.A Planned Unavailability of Consumption Units 
7.1.B Changes in Actual Availability of Consumption Units 
10.1.A Planned Unavailability in the Transmission Grid 
10.1.B Changes in Actual Availability in the Transmission Grid 
Outages 10.1.C Changes in Actual Availability of Off-shore Grid Infrastructure 
15.1.A Planned Unavailability of Generation Units 
15.1.B Changes in Actual Availability of Generation Units 
15.1.C Planned Unavailability of Production Units 
15.1.D Changes in Actual Availability of Production Units 
13.1.A Redispatching 
in om 13.1.B Countertrading 
13:1:€ Costs of Congestion Management 
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5.2 Annex 3 - Questionnaire for online user survey (Output 1) 


Part I: Introduction 


e Which datasets from the Transparency Platform have you used? 

e Do you rely on Transparency data to make business decisions? 

e What do you use Transparency data for? [Fundamental power system 
modelling/Econometric analysis/Statistical analysis/Other (write-in)] 

e Approximately how frequently do you download data from the Transparency Platform? 
[Rarely—I do so a couple of times each year/Regularly—I do so a couple of times each 
month/Frequently—I do so several times each day/Other (write-in) ] 

e How experienced are you with analysing (downloaded) Transparency Platform data? [Not 
very—I use the data rarely/Somewhat—I use the data on a monthly basis/Very—I use 
the data everyday/Other (write-in) ] 


Part II: Completeness 


e Are there missing observations or gaps in the data? [There are many gaps/There are 
some gaps/There are no gaps/I'm not sure] 

e Please specify any incompleteness issues regarding gaps in the data by time series 
and/or geographic area. 

e Are there any types of data not currently available that you would like to see provided 
on the Transparency Platform? 


Part III: Accuracy 


e Do you find data on the platform to be accurate (correct)? [Most values seem 
implausible/Some values seem implausible/Data seems correct/I'm not sure] 

e Please specify any inaccuracies and to which data they are related. 

e Do you find Transparency Platform data to be inconsistent with other sources? If so, 
which data and which other sources? 


Part IV: Timeliness 


e Within what timeframe do you need electricity market data? [Intraday/Within one 
week/Within one month/Other (write-in)] 

e Do you find data on the platform to be available when you need it? [Data is rarely 
available when I need it/Data is usually available when I need it/Data is always available 
when I need it/I'm not sure] 

e Please specify any timeliness issues and to which data they are related. 

e Are historical data being updated with more recent data? 

e Are data updated in a way such that useful legacy data are overwritten? 

e Please specify any issues with updates and to which data they are related. 
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Part V: User friendliness 


e Is finding data on the Platform unintuitive or intuitive [scale of 1-5]? 

e Do you have any suggestions for making the Platform more user friendly? 

e Do you find server response waiting times to be slow or fast [scale of 1-5]? 

e Are you aware of the following options for accessing data (Website GUI, FTP server, 
Restful API, Data repository, Subscriptions, Web services, ECP)? [Not aware of/Aware of 
but have not used/Have used] 

e Why did you choose your current method of accessing the data? [Only option I was 
aware of/Other (write-in)] 

e Please rate the usefulness of the following methods for accessing data: website GUI, FTP 
server, Restful API [scale of 1-5]. 

e Linked here is the data documentation. Were you already aware of this documentation? 

e Do you find the documentation to be of sufficient quality? 

e Is there something missing from the data documentation? 

e Are you aware of the data licence for information obtained from the Platform? 

e Has data licensing prevented you from using the data for any purpose? 


Part VI: Wrapping up 


e What suggestions do you have for Neon regarding improving the Platform? 

e Any additional comments or concerns? 

e How experienced would you consider yourself in using the Transparency Platform? 
Limited experience or expert [scale of 1-5]? 

e Do you have any suggestions of other Platform users who might be interested in joining 
us for an interview? 

e Type of institution [Research/Consulting/Industry/NGO or journalism/Other (write-in)] 
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5.3 Annex 4 - Survey questionnaire for Output 3 (with electricity 


producers and TSOs) 
Dear Sir or Madam, 


The Directorate-General for Energy of the European Commission has mandated VVA 
Consulting, Copenhagen Economics and Deloitte Consulting to carry out a study on the 
quality of electricity market data of transmission system operators, electricity supply 
disruptions, and their impact on the European electricity markets. As part of the study, 
the evaluation team seeks to gather data on outages of generation units by Member 
State and generation technology for the period 2010-2016. 


We have downloaded all the data available on the unavailability of production and 
generation units on the ENTSO-E Transparency Platform, where we have identified a 
number of data gaps. For the purposes of the study, we would like to know if you have 
stored historical data and are in a position to provide detailed data on the most relevant 
variables for each outage, namely: 

e Information on plants (name, location); 
Nature (planned or unplanned); 
Cause(s) 
Length in time; 
Reduction of the availability capacity (in MW); 
Amount of non-generated electricity (in MWh). 


If you do not have such data, we would greatly appreciate it if you could answer the 
following four high-level questions: 


1) What is the share of planned outages in total outages in your Member State? (in 
%, 2010-2016 on a yearly basis) 

2) What is the share of unplanned outages in total outages in your Member State? (in 
%, 2010-2016 on a yearly basis) 

3) Did any of these outages lead to supply disruption events? If yes, could you please 
provide further information on their date, length, the reduction of available 
capacity and the amount of non-generated electricity? 

4) Are you aware of any case where a malicious attack caused an outage in a 
generation unit? 
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5.4 Annex 5 - Survey questionnaire for Output 4 (with national 


regulators and TSOs) 
Dear Sir or Madam, 


The European Commission mandated VVA Consulting, Copenhagen Economics and 
Deloitte to carry out a study on the quality of electricity market data of transmission 
system operators, electricity supply disruptions, and their impact on the European 
electricity markets. As part of the study, this survey aims at gathering data on electricity 
supply disruptions in each Member State on a monthly basis for the period 2010-2016. 


We have downloaded all the electricity supply disruption data available in the ENTSO-E 
Monthly Reports and from the CEER Benchmarking reports. We would like to ask you to 
take part in the following survey to close the relevant data gaps identified on the extent 
and sources of electricity supply disruptions in Europe. 


Survey questions 


The first part of the survey questions will focus on the macro-level with questions on the 
overall number of supply disruptions and the causes for these disruptions. We would be 
very grateful if you could answer those questions in priority. 


The second part of the survey will ask questions at the micro-level to have more 
information on the specific supply disruption events. 


Part 1: Macro-level questions 
Introduction: 


Disruptions are the focus of this survey. A “disruption” of power supply can occur at the 
level of a line, a transformer station or a substation. In the case of a disruption, a certain 
amount of energy will be unavailable during a period. A disruption can lead in some cases 
to a brownout and impact the amount of energy supplied to consumers. 


In some questions, we will ask about outages. An “outage” can occur at the level of the 
electricity generation or the production plants. The outage can be either planned (e.g. 
maintenance) or unplanned (e.g. weather conditions) and will cause a loss of power, 
leading in some cases to disruptions. 


e Country / area of operation 

e Over the period of 2010 to 2016, what was the System Average Interruption 
Duration Index (SAIDI) (monthly figures, if possible)? 

e Over the period of 2010 to 2016, what was the System Average Interruption 
Frequency Index (SAIFI) (monthly figures, if possible)? 

e Over the period of 2010 to 2016, what were the most frequent reasons for supply 
disruptions? 

e Over the period of 2010 to 2016, were any significant supply disruption events 
caused by malicious attacks? 

e Over the period of 2010 to 2016, at which infrastructure level did significant 
disruptions events occur more frequently: TSO- or DSO-level? 
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At TSO-level, most frequently in (a) electric cables/lines, (b) transformer 
stations or (c) interconnectors? 

At DSO-level, most frequently in (a) electric cables/lines or (b) transformer 
stations? 


e Over the period of 2010 to 2016, were some of the significant supply disruption 
events linked to outages in the electricity generation/production plants (i.e. lack of 
power)? If yes, how often did that happen? 


Part 2: Questions on specific disruption events 


e Over the period of 2010 to 2016, what where the 10 most significant supply 
disruptions in your area of operation? For each of the disruptions, please provide: 


o 


o 


start and end date (dd/mm/yyyy, hh:mm) 

duration 

infrastructure elements (cable/line, interconnector, transformer station) 
cause (natural hazard, lack of power, ...) 

reduction in available capacity (MW) 

non-transferred electricity (MWh) 


did the disruption lead to a brownout, i.e. were consumers out of power? 
(yes/no) 


electricity non-supplied to consumers (MWh) 
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5.5 Annex 6 -Survey questionnaire for Output 5 (with TSOs) 
Dear Sir or Madam 


The European Commission mandated VVA Consulting, Copenhagen Economics and 
Deloitte to carry out a study on the quality of electricity market data of transmission 
system operators, electricity supply disruptions, and their impact on the European 
electricity markets. As part of the study, this survey aims to gather data on voluntary 
demand curtailment mechanisms and their use in case of significant electricity supply 
disruptions in the 28 Member State during the period 2010-2016. 


Survey: 


We would like to know whether voluntary demand curtailment mechanisms are in place in 
your country, and in particular whether they have been used in cases of significant supply 
disruptions, meaning when electricity demand exceeded supply and when consumers 
were inevitably going to be cut out of power. We are therefore focusing on voluntary 
demand curtailment agreements but not on price-based demand response in the 
balancing market. 


Questionnaire A: Questions for the TSOs of Belgium, Ireland, Finland, France, Germany, 
Great Britain, Greece, Italy, Netherlands, Poland, Portugal, Spain (where we have pre- 
identified voluntary demand curtailment mechanisms) 


During our initial desk research, we have found that such voluntary demand curtailment 
mechanisms exist in your country. 


1. Which are the main types of electricity consumers involved in these voluntary 
demand curtailment agreements? 

2. How often and when have such voluntary demand curtailment mechanisms been 
activated during significant electricity supply disruptions since 2010? 

3. Do you consider that such voluntary demand curtailments are effective to prevent 
or reduce electricity supply disruptions? How much energy has been delivered to 
other consumers as a result of voluntary demand curtailment avoiding a 
brownout? 

4. Do you plan to increase the part of voluntary demand curtailment mechanisms in 
the future as part of your business plan, or if not in place yet, do you plan to 
introduce such mechanisms? 

Questionnaire B: Questions for all other TSOs 


1. Have such voluntary demand curtailment mechanisms been in place in your 
country in the 2010-2016 period to deal with significant supply disruptions where 
consumers would need to be curtailed anyway? 

2. If yes, which are the main types of electricity consumers involved in these 
voluntary demand curtailment agreements? 

3. How often and when have such voluntary demand curtailment mechanisms been 
activated during significant electricity supply disruptions since 2010? 


4. Do you consider that such voluntary demand curtailments are effective to prevent 
or reduce electricity supply disruptions? How much energy has been delivered to 
other consumers as a result of voluntary demand curtailment avoiding a 
brownout? 

5. Do you plan to increase the part of voluntary demand curtailment mechanisms in 
the future as part of your business plan, or if not in place yet, do you plan to 
introduce such mechanisms? 
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5.6 Annex 7 - List of stakeholders 


Table 20 : List of stakeholders interviewed for Output 1 


Name Institution Sector 

Jan Abrell ETH Zurich Academia 
Lissy Langer TU Berlin Academia 
Jens Weibezahn TU Berlin Academia 
Florian Ziel University Duisburg-Essen | Academia 
Lothar Rausch Oko-Institut Consulting 
Paul-Frederik Bach Freelance consultant Consulting 


Philip Hewitt EnAppSys Data service provider 

Olivier Corradi Tomorrow Data service provider 

Talia Parisi Genscape Data service provider 

Ralf Uttich RWE Industry 

Christian Barwolf LEAG Industry 

Jens Wimschulte Vattenfall Industry 

Chris Münster Vattenfall Industry 

Tobias Schulz Vattenfall Industry 

Sigurd Pedersen DONG Energy Industry 

Dave Jones Sandbag NGO 

Antonella Battaglini Renewables Grid Initiative | NGO 

Thorsten Lenck Agora Energiewende NGO 

Mara Marthe Kleiner Agora Energiewende NGO 

Rafael Muruais-Garcia ACER Policy 

Marcus Mittendorf EEX Power exchange & data service 
provider 

Katrin Petri EEX Power exchange & data service 
provider 

Filippo Pirovano EDF Trading Trading 
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Table 21 


Country 
BE 
Belgium 


: List of stakeholders responding to the survey of Output 1 who provided 
biographical information 


CH 
Switzerland 
DE 
Germany 


DK 
Denmark 


OHertz 
Agora Energiewende 
Frankfurt Institute for 
Advanced Studies 
Fraunhofer IWES 
Hochschule Niederrhein 
Oko-Institut 
echnische Universitat Berlin 
echnische Universitat 
Dresden 
niversitat Duisburg-Essen 
Zentrum fur Sonnenenergie- 
nd Wasserstoff-Forschung 
Baden-Württemberg 
ZNES Flensburg 
Danmarks Tekniske 
niversitet 
Dansk Energi 


Independent consultant 
PM Energy 


[Type of Institution |Number of stakeholders| 


Research 


Academic 


Academic 


Research 


Research 


Academic 


Interest organisation 
onsulting 


1 


Ea Energianalyse 


Finland 

France 

; 
Research Institute 


NL Delft University of Technology Academic 1 
Netherlands 


European Commission overnment NENNEN! 


1 
railStone Group [ndustry 1 


Ireland 
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Table 22 : List of stakeholders contacted for the survey of Outputs 3, 4, 5 
Country Main electricity producers National regulators Main DSOs 


AT Verbund AG Energie Control Austria (E- 
Austria ontrol) i 
Netz Niederósterreich 
mbH (NO Netz) 
B 
B 


Electrabel ommission de Régulation |Elia System Operator SA ORES, Tecteo (Resa), 
Belgium de l'Electricité et du Gaz (Elia) Régie de Wavre, AIESH 
and AIEG in Wallonia 
ibelga in the Brussels- 
apital Region 
Eandis and Infrax in 
Flanders 


E 
G Natsionalna Elektricheska [State Energy & Water Electroenergien — Sistemen|CEZ 
Bulgaria Kompania EAD (NEK) Regulatory Commission Operator EAD (ESO) 
(SEWRC) EVN 
Energo-Pro 
Y 
Z 


C Electricity Authority of yprus Energy Regulatory Cyprus Transmission Electricity Authority of 
Cyprus Cyprus Authority (CERA) ystem Operator (Cyprus yprus (EAC) 


C CEZ.a.s rgeticky ční Ur .S. EZ.a.s 
Czech Republic 

Energy Regulatory Office 

(ERO) 


DE EnBW, E.On Federal Network Agency for | TransnetBW GmbH 
Germany Electricity, Gas, (TransnetBW) 
elecommunications, Posts 
and Railway TenneT TSO GmbH 
(Bundesnetzagentur - (TenneT DE) 
BNetzA) 
Amprion GmbH 
(Amprion) 
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Country Main electricity producers National regulators | TSOs | Main DSOs | 


mbH (50Hertz 
DONG Energy Energitilsynet - Danish Energinet.dk Dansk Energi 
DERA 


Public Power Corporation PuOuirorikr] Apxn Evépyeiac / | Independent PowerDEDDIE 
Greece S.A. Regulatory Authority for ransmission Operator S.A. 
Energy (PAE / RAE) (IPTO) .S. 


D 

EE 

EL 

ES Endesa Agencija za energijo / Red Eléctrica de España Iberdola 

Spain Energy Agenc .A. (REE 

FI Fortum, Pohjolan Voima Energiavirasto - Energy Fingrid Oyj (Fingrid) Energia 

Finland Authorit 

FR EDF ommission de Régulation | Réseau de Transport Enedis 

France de l'Energie (CRE) d'Electricité (RTE) 

G EDF Energy, Scottish and Office of Gas and Electricity | National Grid Electricity Energy Networks 


United Kingdom Southern Energy plc ransmission plc (National 
rid) 


K 
Eesti Energia AS Elering AS (Elering AS) Elektrilevi OÜ 
Estonia 
B 


ystem Operator for 
Northern Ireland Ltd (SONI) 


Scottish Power 
ransmission plc 
(SPTransmission) 


HR HEP Group Hrvatska energetska HOPS d.o.o. (HOPS) HEP ODS 
Croatia regulatorna agencija (HERA 
HU 


MVM Hungarian Electricity Magyar Energetikai és MAVIR Magyar|ELMU Net Ltd 
Hungary Ltd. Kôzmü-szabälyozäsi Hivatal Villamosenergia-ipari 
(MEKH) Atviteli 
Rendszerirányító ZärtkôrüenEMASZ Net Ltd 
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Country Main electricity producers National regulators | TSOs | Main DSOs | 


n 7 
LU RE (MAVIR ZR) 

Ireland Regulation (CER 

Ital Elettrica e il Gas (AEEG Nazionale SpA (Terna 


Lietuvos Energijos Gamyba alstybiné kainu ir Litgrid AB (Litgrid) AB LESTO 
Lithuania energetikos kontrolés 
komisija / 
National Control 
ommission for Prices and 


E 
T 
LT 
LU Enovos Institut Luxembourgeois de |Creos Luxembourg S.A. Creos Luxembourg 
Luxembourg Régulation (ILR Creos Luxembourg 
LV j 
L 
T 
R 


Latvenergo AS abiedrisko pakalpojumu AS Augstsprieguma tikls Sadales tikls AS 
Latvia regulēšanas komisija/ (Augstsprieguma tikls) 
Public Utilities Commission 
(PUC) 
Netherlands Markt (ACM NL 
P Polska Grupa Energetyczna Urzad Regulacji Energetyki / | Polskie Sieci GKPGE 
Poland Energy Regulatory Office Elektroenergetyczne S.A. 
(URE / ERO) (PSE S.A.) 
P EDP - Energias de Portugal,  |Entidade Reguladora dos Rede Eléctrica Nacional, EDP 
Portugal S.A. erviços Energéticos / .A. (REN) Distribuicgao, Energia S.A 


O Electrocentrale Bucharest Antoritatea Nationalá de C.N. Transelectrica S.A. ENEL Romania 
Romania Reglementare in domeniul  (Transelectrica) 


SE Vattenfall AB Energimarknadsinspektionen| Svenska kraftnat Vattenfall AB 
E pue a pem 7 
Inspectorate (EI 
Slovenia (HSE) Energy Agency 
Slovak Republic odvetví (URSO) / prenosová sústava, a.s. 
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Country Main electricity producers National regulators | TSOs | Main DSOs 


Regulatory Office for (SEPS) SSE (Central) 
Network Industries (RONI) 
SE (East 
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